acceptodds
Under review as a conference paper at ICLR 2027

Iterative Self-Improvement with World-Expectation Optimization for World-Action Model

Abstract

Robot foundation models have enabled increasingly general manipulation through large-scale pretraining. Recent world action models (WAMs) further unify action generation and future prediction, providing a foundation for robots to learn both how to act and what their actions cause. As robots interact with their environments, each execution provides new evidence about the effectiveness of their actions and the accuracy of their predictions, creating an opportunity for continued self-improvement beyond pretraining. However, existing self-improvement approaches often evolve policies with external predictive models, without directly improving predictive capabilities within the action-generating model itself. We introduce World-ISI, an iterative framework that turns this execution feedback into the joint evolution of action generation, world prediction, and value estimation within a single WAM. To leverage the WAM’s own predictions of future task outcomes during self-improvement, we further propose World-Expectation Optimization (WEO). WEO uses the difference between observed returns and the model’s own value predictions to weight learning from successful actions, while using failed executions in dedicated world- and value-prediction modes and retaining auxiliary prediction losses on successful policy samples. Applied to Cosmos Policy, World-ISI improves average success on 24 RoboCasa tasks from 66.7% to 70.1% after four iterations. In a controlled comparison at 6,000 training steps, WEO achieves 69.0% success versus 67.5% for standard supervised fine-tuning, supporting the effectiveness of learning from the model’s own expectations.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Robot Learning, Iterative Self-Improvement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GWjiOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes World-ISI, an iterative self-improvement framework for world-action models (WAMs) that jointly updates action generation, world prediction, and value estimation using self-collected interaction data. The authors introduce World-Expectation Optimization (WEO), which weights successful action trajectories based on the difference between observed returns and the model's predicted values, while using failed trajectories for world and value supervision. The framework is evaluated on 24 RoboCasa tasks and real-world WidowX and ALOHA robots. Results show that World-ISI improves the average RoboCasa success rate from 66.7% to 70.1% over four iterations, along with improvements in future prediction quality and real-world manipulation performance.

Critical strengths
  1. Promising self-improvement framework. I appreciate the idea of jointly updating action, world, and value predictions using self-collected experience, especially the use of failed trajectories for prediction supervision.

  2. Well-structured comparison of optimization strategies. The authors compare WEO with SFT using the same fixed dataset and training budget, achieving 69.0% vs. 67.5% success at 6k steps. This helps distinguish the effect of the proposed optimization from additional data collection.

  3. Encouraging empirical results. World-ISI improves RoboCasa success from 66.7% to 70.1%, with additional improvements in world prediction metrics and real-world robot experiments.

Critical weaknesses
  1. Limited evaluation of self-improvement under evolving task distributions. Although World-ISI demonstrates consistent improvements across iterations and includes some OOD evaluations, the iterative training remains focused on the same 24 RoboCasa tasks. It is unclear whether the framework can continually adapt to new tasks, changing visual conditions, or environment dynamics while retaining previously learned capabilities. A stronger continual learning evaluation would make the self-improvement claim more convincing.

  2. Insufficient evidence for the contribution of world and value prediction. The auxiliary-loss ablations also change training sampling ratios (80/10/10 vs. 100/0/0), making it difficult to isolate the benefits of prediction supervision. Moreover, value prediction accuracy and calibration are not directly evaluated, despite their central role in WEO. The paper therefore provides limited evidence of how improved world/value modeling contributes to policy improvement.

  3. Confounding between expectation weighting and action-loss scaling. Under binary rewards, WEO assigns weights ≥ 1 to all successful samples, increasing the overall action-loss contribution relative to world/value losses. Thus, its improvement over SFT may partly result from stronger action supervision rather than expectation-aware weighting. A mean-weight-matched SFT baseline is needed to distinguish these effects.

Questions for authors
  1. Self-improvement and generalization. Can the authors provide stronger evidence of self-improvement under evolving task distributions, such as introducing new tasks, visual variations, or changing dynamics across iterations? How well does the model retain previously learned capabilities?

  2. Effectiveness of world and value prediction. Figure 5 shows that adding world and value losses improves policy success, but it remains unclear how these prediction capabilities contribute to the improvement. Can the authors provide more direct evidence, especially the accuracy or calibration of value predictions across iterations? A controlled ablation with matched policy-sample exposure would also help clarify the contribution of auxiliary prediction supervision.

  3. Effectiveness of WEO weighting. Can the authors compare WEO with a constant-weight SFT baseline matched to WEO's average action-loss weight? This would help clarify whether the improvement comes from expectation-aware weighting or simply stronger action supervision.

I am open to raising my rating if the authors can convincingly address these concerns, particularly regarding the contributions of world/value prediction and the effectiveness of WEO.

6 comments
AuthorsOpenReview ↗

Authors' Response to Reviewer GWji

We thank the reviewer for their thoughtful evaluation and constructive feedback. We appreciate the recognition of our self-improvement framework, the effectiveness of WEO compared to SFT, and the encouraging results across simulation and real-world robotic tasks. We address the reviewer's questions as follows.

Q1:

  1. Generalization under distribution shifts. In RoboCasa, we evaluate on held-out object instances across five kitchen scenes per task, including two with visual styles absent from training (Sec. 4.1). On WidowX, we collect demonstrations and iterative rollouts only in the standard training configurations and reserve the variations for testing. For the hotpot task, we test with different ingredients or rearrange familiar ones. For the brick task, we vary the bricks, the bowls, or the brick placements. After two iterations, performance improves in both hotpot settings and with different bricks and bowls, while performance on unfamiliar brick placements returns close to its initial level. On ALOHA, we test vegetable transfer by swapping the cucumber and eggplant positions or replacing the cucumber with a carrot. The cucumber–eggplant completion score rises from 39.6 to 66.2, while the carrot–eggplant score remains at 59.6 (Sec. 4.5). These experiments show where improvements transfer to held-out conditions. We acknowledge that continual adaptation to new tasks or changing dynamics during training remains an important direction for future research, and we are committed to further investigating these challenges in our future work.

  2. Retention of previously learned capabilities. We additionally evaluated the base policy and all four World-ISI checkpoints on the same 240 in-domain RoboCasa initial conditions across the 24 tasks, keeping object instances and initial conditions fixed across checkpoints. Every iteration exceeded the base success rate by 2.5–6.3 percentage points. We have incorporated these results into Appendix A.2 (Table 5) of our manuscript.

MetricCosmos BaseIter. 1 (3k)Iter. 2 (4k)Iter. 3 (5k)Iter. 4 (6k)
In-domain success rate65.8%68.3%69.6%72.1%69.2%

Q2:

  1. Does value prediction improve across iterations? We measured the training MSE between the predicted and target value latents on value-supervised samples. During the initial 3,000 steps, this loss fell from 0.433 to 0.000160, comparing the first and last 100 steps. It also decreased within each subsequent iteration under the same measurement protocol. These decreases show that the model fits its value targets more closely during each round.
Training intervalFirst 100 stepsLast 100 steps
Iter. 1 → Iter. 20.0004190.000124
Iter. 2 → Iter. 30.0003550.000135
Iter. 3 → Iter. 40.0001890.000121
  1. Why use a 100/0/0 policy/world/value sampling ratio for the policy-only baseline? The table below shows which losses are active for each sample type. Policy samples come from successful rollouts, while dedicated world-model and value samples come from failed rollouts. Within each cell, the three entries correspond to the action, future-state, and value losses, respectively. “Main” and “Aux.” denote primary and auxiliary objectives, and a dash denotes an inactive loss.
Sample typeFull WEOW/o value lossW/o world or value loss
PolicyMain / Aux. / Aux.Main / Aux. / —Main / — / —
World model— / Main / Aux.— / Main / —— / — / —
Value— / — / Main— / — / —— / — / —

We respectfully clarify that when both the world-model and value losses are disabled, only policy samples have an active loss. Continuing to draw world-model or value samples would therefore use part of the batch for samples that contribute zero gradient. We use 100/0/0 sampling so that every sample contributes to the action objective. At the same 3,000-step training budget, this gives the policy-only baseline 25% more policy samples than WEO receives with 80/10/10 sampling (2.40M versus 1.92M). We have added this explanation and the loss breakdown to Appendix A.5 (Table 7).

  1. Does the policy gain persist when policy-sample exposure is matched? Yes. We added a control that matches the number of policy samples at 1.92M. WEO reaches this count in 3,000 steps with 80% policy sampling, and the policy-only baseline reaches it in 2,400 steps with 100% policy sampling. At matched exposure, WEO achieves 67.4% success compared with 66.4% for the policy-only baseline. We have incorporated these results into Appendix A.5 (Table 8) of our manuscript.
Method (action / WM / value)Policy samplesTraining stepsSuccess rate
WEO (80/10/10)1.92M3,00067.4%
Policy only (100/0/0)1.92M2,40066.4%
Policy only (100/0/0) (paper)2.40M3,00066.6%
AuthorsOpenReview ↗

Authors' Response to Reviewer GWji (2)

Q3:

We thank the reviewer for suggesting this control. WEO does not normalize its action weights to have unit mean. We trained a constant-weight SFT baseline using the same fixed dataset, sampling procedure, and a 3,000-step training budget as WEO. We assigned every successful action sample a weight of 3.7176, the estimated mean action weight under WEO. Results are shown in the table below.

MethodConstant-weight SFTWEO
Success rate66.1%67.4%

WEO outperformed the constant-weight baseline by 1.3 percentage points, suggesting that expectation-aware weighting contributes to policy improvement even when the average action weight is matched. We have incorporated these results into Appendix A.5 of our manuscript.

Thanks for the clarification and additional experiments. I am satisfied with the responses to Q1 and Q3. For Q2, the matched policy-sample comparison also helps address my concern about sampling.

My remaining concern is about value prediction. The lower training MSE shows better fitting of the training targets, but it does not show whether the values used by WEO are accurate on new rollouts. Value training uses observed futures, while WEO uses predicted futures during collection. Could you evaluate these pre-execution value predictions against actual outcomes on separate test rollouts across iterations?

Reviewer_asgeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes World-ISI, an iterative self-improvement framework for a unified world-action model, with a training objective called World-Expectation Optimization (WEO). World-ISI jointly evolve action generation, world prediction and value estimation within a single WAM by using the difference between observed returns and the model's own value predictions to weight learning from successful actions. This leads to performance improvements across a lsit of robotics tasks and scenarios.

Critical strengths

The paper is well written and easy to follow. The objective is clear, and jointly update action generation, world prediction, and value estimation within a unified world action model sounds interesting.

Critical weaknesses
  1. Unclear novelty. The core of WEO, which is equation (1), is quite similar to advantage-weighted regression (AWR). Using a world models imagined future to derive a learning signal is also well established (e.g. Dreamer). It would be helpful to state precisely which compnent is new, and point to experiments that show this new design matters.
  2. Missing baseline comparison. The paper claims that improving prediction within the model may be better than using external value or world models. The paper lists other baselines such as VLAW, Q-Planning and RISE, but none of them are directly compared. Also, no error bars, standard deviations or confidence intervals are given, which makes it hard to tell whether the improvements are statistically significant or just by chance.
Questions for authors

N/A

3 comments
AuthorsOpenReview ↗

Authors' Response to Reviewer asge (1)

We thank the reviewer for their thoughtful evaluation and feedback on our work. We appreciate the recognition of the clear motivation and the potential of jointly improving action generation, world prediction, and value estimation within a unified world-action model. We address the reviewer's questions as follows.

W1:

Comparison with AWR. We respectfully clarify two differences between AWR and World-ISI.

  1. Unified optimization. AWR improves the policy through advantage-weighted regression using a separately trained value critic. In contrast, World-ISI jointly improves action generation, world prediction, and value estimation within a single WAM through WEO. It uses the WAM's own pre-execution expectations to weight policy learning, while observed future states and returns supervise world and value predictions. All three objectives update shared model parameters, enabling execution feedback to improve both the policy and its own predictive capabilities without requiring a separate critic.

  2. Policy weighting. AWR computes G_λ(t) − V(s_t), where G_λ(t) is the TD(λ) return estimate following the action at time t, and V(s_t) is the critic’s expected return from the pre-action state s_t. Our World-Expectation Optimization (WEO) uses G_t − v̄_t, where G_t is the observed Monte Carlo return and v̄_t is the model’s pre-execution return forecast for the proposed action chunk and its imagined outcome. WEO therefore gives greater imitation weight to successful actions whose outcomes exceeded the model’s forecast.

We adapted AWR to Cosmos Policy using a fixed dataset of successful and failed RoboCasa trajectories. Following Peng et al., we trained a separate state-value critic on TD(λ) returns with λ = 0.95 and computed each decision’s advantage by subtracting its pre-action state value from its estimated return. Following the authors’ released implementation, we standardized the advantages and exponentiated them with temperature 1.0, clipping the weights at 20. We applied these weights to the Cosmos action-denoising loss over the executed portion of each 32-step action chunk, up to 16 steps. We used the paper’s schedule of 200 critic updates and 1,000 policy updates per round, for three rounds. At the same 3,000 policy-training steps, WEO outperformed Cosmos+AWR on 24 RoboCasa tasks. We have incorporated these results into Table 1 of our manuscript.

MethodPnPOoCOthersAverage
Cosmos+AWR48.8%86.7%58.4%62.3%
World-ISI (Iter. 1, 3k)51.6%93.7%64.4%67.4%

Comparison with Dreamer. Both Dreamer and our World-ISI collect experience through repeated interactions. The main differences are policy supervision and model architecture.

  1. How imagined and observed returns guide policy learning. Dreamer uses that experience to learn a world model and trains its actor on returns predicted along imagined trajectories. World-ISI targets the improvement of a pretrained robot manipulation policy from its own executions. In WEO, imagination supplies a forecast, and execution in the simulated environment supplies the observed return. The gap between the two determines how much weight the executed action receives. This gives WEO more direct supervision about the outcome of each execution: even when the model predicts a low return, an observed success allows WEO to give that action greater weight during fine-tuning.

    To test whether this weighting adds value beyond repeated data collection, we added an experiment to compare World-ISI with iterative SFT. Both collect new rollouts under the same collection targets and jointly train action, world, and value prediction. Iterative SFT gives successful actions a constant imitation weight. World-ISI uses WEO’s execution-grounded weights and achieves higher mean success at all four iterations under three evaluation seeds. We have incorporated these results into Table 3 of our manuscript.

    MethodIter. 1Iter. 2Iter. 3Iter. 4
    Iterative SFT64.7%66.6%61.8%67.6%
    World-ISI (Ours)67.4%67.9%69.2%70.1%
  2. Model architecture. Dreamer uses a recurrent latent world model with separate actor and critic networks and updates the actor by backpropagating through imagined trajectories. World-ISI builds on a single WAM that jointly generates actions, predicts their visual consequences, and estimates their values. Because these functions share parameters, the observed consequences and returns from robot executions directly supervise the model that generates the next actions. Our RoboCasa ablation (Fig. 5) shows the benefit of this joint training for manipulation.

AuthorsOpenReview ↗

Authors' Response to Reviewer asge (2)

W2:

  1. Baseline comparison with external value or world models. Thank you for suggesting baseline comparison. We adapted VLAW to Cosmos Policy. Following VLAW, we fine-tuned a Qwen reward model on 250 labeled RoboCasa rollouts across 24 tasks, used it to filter 2,500 trajectories imagined by the Cosmos world model, and fine-tuned Cosmos Policy for 3,000 steps. Results of the same optimization steps are shown in the table below. We have incorporated these results into Table 1 of our manuscript.

    MethodPnPOoCOthersAverage
    Cosmos+VLAW47.3%90.0%59.4%63.0%
    Cosmos+WEO51.6%93.7%64.4%67.4%
  2. Statistical reliability. Regarding statistical reliability, the RoboCasa policy success rates reported below are averaged over three evaluation seeds. We provide the per-seed results and standard deviations below.

    RoboCasa MethodSeed 195 (%)Seed 196 (%)Seed 197 (%)Mean ± SD (%)
    Pretrained Cosmos Policy66.1768.0866.0066.75 ± 1.16
    World-ISI (Iter. 1)68.2567.1766.9267.44 ± 0.71
    World-ISI (Iter. 2)67.0069.2567.3367.86 ± 1.21
    World-ISI (Iter. 3)70.1768.0069.3369.17 ± 1.09
    World-ISI (Iter. 4)70.6769.4270.2570.11 ± 0.64

Please take a moment to respond to the rebuttal. Thank you!

Reviewer_LRi3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

World Action Models (WAMs) can acquire experiences during execution, which can be utilized to improve their action generation and prediction capabilities. The authors introduce an iterative framework named World-ISI, which leverages execution feedback, including observed returns, executed actions, and future observations, to improve the prediction and decision-making capabilities of WAMs. Based on this framework, the authors propose World-Expectation Optimization (WEO), which uses the difference between observed returns and value predictions to weight the policy imitation loss, encouraging the learning of successful actions whose actual returns exceed expectations. By jointly optimizing the weighted policy loss and unweighted world and value prediction losses, WAMs can leverage execution experiences for self-improvement, achieving higher success rates on RoboCasa and real-world WidowX and ALOHA tasks.

Critical strengths
  1. Self-improvement is a promising direction for enhancing the capabilities of pretrained WAMs. The proposed World-ISI framework provides a practical approach to leveraging execution experiences for jointly improving action generation, world prediction, and value estimation.
  2. The experiments cover RoboCasa simulation tasks and real-world WidowX and ALOHA platforms, demonstrating the effectiveness of the overall self-improvement framework across different robotic embodiments and experimental settings.
  3. WEO provides a simple and practical approach to policy improvement by leveraging the value predictions already available within the pretrained WAM, without introducing complex additional modules.
  4. The paper is clearly written and well organized.
Critical weaknesses

1.The World-ISI framework follows a standard iterative data collection and policy improvement paradigm, while the proposed WEO objective closely resembles advantage-weighted regression (AWR), where the difference between observed returns and predicted values is used to weight the imitation loss. Although applying these ideas to a unified WAM is practically meaningful, the paper's core contribution appears to be primarily an adaptation of existing concepts to a new setting, rather than a fundamentally novel methodological development.
2. Although the paper separately evaluates WEO against SFT on fixed data and iterative WEO against non-iterative WEO, the experiments lack an iterative SFT baseline that uses newly collected rollouts under a matched interaction and training budget. This comparison is important for determining whether the proposed expectation-based weighting provides additional benefits over standard iterative imitation learning and isolating the contribution of iteratively collected rollout data. Without this baseline, it remains unclear whether iterative SFT could achieve comparable performance or potentially outperform World-ISI with additional iterations.

Questions for authors
  1. Could the authors provide an iterative SFT baseline on RoboCasa to better isolate the effectiveness of WEO within the World-ISI framework?
  2. Whether the weights in WEO are normalized. If not, could the authors clarify whether the observed performance improvements are attributable to the proposed weighting mechanism or simply to changes in the overall scale of the policy learning loss?
3 comments
AuthorsOpenReview ↗

Author's response to Reviewer LRi3

We thank the reviewer for their thoughtful evaluation and positive feedback on the motivation, practical design, experimental validation, and clarity of our work. We appreciate the recognition of World-ISI's potential for self-improvement in world-action models. We address the reviewer's questions as follows.

W1:

We respectfully clarify two differences between AWR and World-ISI.

  1. Policy weighting. AWR computes G_λ(t) − V(s_t), where G_λ(t) is the TD(λ) return estimate following the action at time t, and V(s_t) is the critic’s expected return from the pre-action state s_t. Our World-Expectation Optimization (WEO) uses G_t − v̄_t, where G_t is the observed Monte Carlo return and v̄_t is the model’s pre-execution return forecast for the proposed action chunk and its imagined outcome. WEO therefore gives greater imitation weight to successful actions whose outcomes exceeded the model’s forecast.

  2. Unified optimization. Our World-ISI jointly optimizes action generation, world modeling, and value prediction within a single WAM. AWR uses a separate value model to guide the policy update. In World-ISI, the same WAM proposes an action, predicts its consequences, and evaluates the imagined outcome. During training, execution feedback weights the imitation of successful actions, while observed consequences and returns supervise the WAM’s world and value predictions. Each new collection round therefore uses a WAM whose policy, world model, and value estimator have been updated together.

To further compare our method with AWR, we adapted AWR to Cosmos Policy using a fixed dataset of successful and failed RoboCasa trajectories. Following Peng et al., we trained a separate state-value critic on TD(λ) returns with λ = 0.95 and computed each decision’s advantage by subtracting its pre-action state value from its estimated return. Following the authors’ released implementation, we standardized the advantages and exponentiated them with temperature 1.0, clipping the weights at 20. We applied these weights to the Cosmos action-denoising loss over the executed portion of each 32-step action chunk, up to 16 steps. We used the paper’s schedule of 200 critic updates and 1,000 policy updates per round, for three rounds. At the same 3,000 policy-training steps, WEO outperformed Cosmos+AWR on RoboCasa evaluation, shown in the table below. We have incorporated these results into Table 1 of our manuscript.

MethodPnPOoCOthersAverage
Cosmos+AWR48.8%86.7%58.4%62.3%
World-ISI (Iter. 1, 3k)51.6%93.7%64.4%67.4%

W2 & Q1:

We thank the reviewer for suggesting this baseline comparison. We have added an iterative SFT baseline on RoboCasa under the same iterative training setting to isolate the contribution of WEO. Iterative SFT also optimizes the same world and value prediction losses, with the only difference being the absence of expectation-based weighting in the policy loss. Across three random seeds, iterative SFT achieves its best average success rate of 67.6% at the fourth iteration, while World-ISI achieves 70.1% at the same iteration, outperforming iterative SFT by 2.5 percentage points. These results demonstrate that the improvements of World-ISI cannot be attributed solely to iterative rollout collection and support the additional effectiveness of expectation-based weighting. We have incorporated these results into Table 3 of our manuscript.

MethodIter. 1Iter. 2Iter. 3Iter. 4
Iterative SFT64.7%66.6%61.8%67.6%
World-ISI (Ours)67.4%67.9%69.2%70.1%

Q2:

We thank the reviewer for the suggestion. WEO does not normalize its action weights to have unit mean. To test whether the average weight alone explains the improvement, we trained a constant-weight SFT baseline that assigns every successful action sample the same weight of 3.7176, the estimated mean action weight under WEO.

MethodConstant-weight SFTWEO
Success rate66.1%67.4%

WEO outperformed the constant-weight baseline by 1.3 percentage points, suggesting that expectation-aware weighting contributes to policy improvement even when the average action weight is matched. We have incorporated these results into Appendix A.5 of our manuscript.

Thank you for your detailed response and clarifications. However, my concerns regarding the limited novelty of the proposed method compared to existing approaches remain. Therefore, I will maintain my current score.

We are glad that our additional experiments and clarifications have addressed W2, Q1 and Q2. We understand that your concern regarding novelty remains, and would like to further clarify two specific methodological distinctions from prior work.

While iterative data collection and policy updates are established practices, our contribution lies in how executions are used to improve a unified WAM.

First, compared with AWR, (1) our method addresses a different optimization problem. AWR focuses on policy improvement through advantage-weighted regression using a separately trained value critic, but does not itself address the joint improvement of action generation, world prediction, and value estimation. In contrast, WEO is designed for a unified WAM, jointly optimizing all three capabilities within a single model using execution feedback. (2) Specifically, while AWR compares estimated returns against a state-value baseline, WEO compares observed returns with the WAM's pre-execution predictions for the proposed actions and their imagined futures, assigning higher imitation weights to unexpectedly successful actions. Meanwhile, observed future states and returns supervise world and value predictions within the same WAM, without requiring a separate critic. Experimentally, WEO outperforms Cosmos+AWR (67.4% vs. 62.3%) and constant-weight SFT (67.4% vs. 66.1%). Our ablations in Figure 5 further show that success increases from 66.6% without world and value supervision, to 66.9% with world supervision, and to 67.4% with both, supporting the benefit of jointly optimizing these capabilities.

Second, compared with existing iterative self-improvement approaches, VLAW alternately trains separate world and policy models using reward-filtered imagined trajectories, while RISE iteratively improves the policy through imagined rollouts with frozen dynamics and value models. In contrast, World-ISI couples policy and predictive learning within a single WAM across iterations: the updated policy collects new experiences, while the updated world and value predictions provide new pre-execution expectations for the proposed actions. The discrepancies between these expectations and actual returns then determine the weights for subsequent policy updates, allowing the WAM's own predictive learning to directly influence how its policy improves. Our adapted Cosmos+VLAW baseline achieves 63.0%, compared with 67.4% for World-ISI at 3,000 training steps. Furthermore, World-ISI outperforms iterative SFT (70.1% vs. 67.6%) under matched interaction and training budgets, supporting the effectiveness of WEO within this iterative framework.

We sincerely appreciate the reviewer's constructive feedback, which has helped us clarify and strengthen our contribution.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions to the Manuscript

We sincerely thank the reviewers for their time and constructive feedback. In response to their comments, we made the following revisions.

  • Section 4.2 and Appendix A.1.1 (additional baselines). We implemented AWR and VLAW on the same Cosmos Policy backbone. After 3,000 policy updates, WEO achieved 67.4% success across 24 RoboCasa tasks, compared with 62.3% for AWR and 63.0% for VLAW (Tables 1 and 3).
  • Section 4.3 and Appendix A.2 (in-domain evaluation). We evaluated the base policy and all four World-ISI checkpoints on the same in-domain initial conditions. World-ISI outperformed the base policy at every iteration, with gains of 2.5 to 6.3 percentage points (Table 5).
  • Section 4.6 (iterative SFT baseline). We added an iterative SFT baseline that follows the same rollout collection and training schedule as World-ISI. World-ISI outperformed iterative SFT in all four rounds, reaching 70.1% success in the final round compared with 67.6% for iterative SFT (Table 3).
  • Appendix A.5 (sampling and action-weight ablations). With the number of policy training samples held fixed, WEO achieved 67.4% success compared with 66.4% for policy-only training (Table 8). To test whether assigning different weights to individual samples matters, we also trained SFT with a constant action weight equal to WEO’s mean. This baseline achieved 66.1%, compared with 67.4% for WEO.

Again, we sincerely appreciate the reviewers’ thoughtful suggestions and look forward to further feedback and discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.