acceptodds
Withdrawn from ICLR 2027

WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Post-training VLA policies typically rely on supervised fine-tuning with costly expert demonstrations or reinforcement learning with expensive and potentially unstable real-world exploration. World models offer a promising alternative by evaluating candidate behaviors through imagined futures, yet effective post-training requires more than accurate prediction: imagination must be scheduled where it is useful, bounded to limit prediction errors, and translated into reliable policy-learning signals. In robotic manipulation, the value of imagination varies substantially across execution stages, while extended rollouts can accumulate prediction errors and introduce unreliable learning signals. We introduce **WISE (World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models)**, a unified framework that coordinates when and how world-model imagination is used during policy refinement. WISE selectively invokes imagination at interaction-relevant states, performs bounded multi-view rollouts, evaluates candidate futures using progress and completion signals, and uses their relative outcomes to refine actions generated from real interaction contexts. Extensive experiments with both and demonstrate consistent improvements across diverse manipulation tasks while reducing GPU computation time by approximately 80% compared with full imagination. Real-world evaluations further show substantial gains in robustness and generalization under diverse real-world distribution shifts.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Post-training, World Model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EvpMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents WISE, a framework for world-model-assisted VLA post-training. It allocates imagination computation to interaction-relevant states, such as grasping and manipulation. A visual scheduler selects states, after which the VLA samples candidate actions and evaluates their consequences through bounded world-model rollouts. A reward model combines progress, confidence and completion signals, while repeated imagination and reliability filtering produce group-relative advantages for updating the VLA action head. Policy supervision uses only the first action chunk sampled from each real interaction context. The framework is evaluated with π0 and π0.5 on MimicGen tasks and real robots.

Critical strengths
  1. The paper addresses an important computational efficiency problem. Generating many candidate futures with a world model is expensive, and WISE explicitly studies how to allocate this computation. In the scheduling ablation across three simulated tasks, it reduces the average number of selected states from 138 to 28 and GPU time from 11.45 to 2.61 hours while achieving higher success rates, demonstrating a favorable balance between performance and computation.

  2. The evaluation covers two VLA backbones and both simulation and real robots. Across five MimicGen tasks, WISE improves the average success rates of π0 and π0.5 by 10.0 and 7.9 percentage points, respectively. Real robot evaluations additionally cover changes in height, lighting, distractors and language instructions, supporting the effectiveness of the overall framework and the robustness of the resulting policies.

  3. The framework explicitly addresses imagination errors and supervision reliability. Bounded rollouts, repeated imagination and candidate-group filtering aim to limit prediction errors and unstable reward estimates. Moreover, imagined trajectories evaluate candidate actions, while policy supervision remains grounded in real interaction contexts and their first action chunks. This may reduce the risks associated with directly learning from unreliable synthetic state–action pairs.

  4. The ablations provide evidence beyond overall performance gains. The scheduling study compares learned, random, and uniform selection with the same number of selected states, supporting the value of state selection itself. Reward ablations also show that combining progress, completion and the additional reward components outperforms individual signals, providing empirical support for the main design choices.

Critical weaknesses
  1. The scheduler’s ability to identify useful imagination contexts remains insufficiently validated. It is trained using weak labels associated with grasping and manipulation, so its benefits may largely come from concentrating computation on these stages. The scheduling ablations cover only Stack, Threading, and Mug Cleanup in MimicGen, where precise alignment and contact interactions are central challenges. The paper lacks comparisons within the same stage and against simple stage rules. Its independent contribution is also not isolated in real robot experiments.

  2. The GRPO and DPO baselines are insufficiently specified. The paper does not clearly explain how these methods are adapted to π0’s flow-matching action generation, including their optimization objectives and computation of policy probabilities or surrogates. Key training configurations and budgets are also unclear. These choices can substantially affect baseline performance, limiting reproducibility and assessment of comparison fairness.

  3. The methodological novelty relative to prior work needs clarification. WMPO already combines world-model rollouts, learned rewards and GRPO for VLA optimization. VLAW explores world-model-assisted VLA post-training. AWM studies advantage-weighted flow-matching updates. WISE adds selective imagination, bounded rollouts and reliability filtering, but its central scheduler is a classifier trained with weak stage labels and used through threshold gating. The paper should clarify the substantive advances beyond integrating these components.

Questions for authors
  1. Does the scheduler identify states where imagination provides greater benefits? In Stack Cube, one state may have the upper cube already aligned, while another places it near the edge, where different actions could lead to stable stacking or toppling. Does the scheduler preferentially select the latter states, and does imagination at these states produce greater training gains?

  2. How does the learned scheduler compare with simple stage rules? Under the same imagination budget, please compare it with random or uniform selection within manipulation stages identified using grasp signals.

  3. What is the scheduler’s independent contribution on real robots? Could the authors provide an equal-budget ablation comparing learned, random and uniform scheduling?

  4. How are π0 + GRPO and π0 + DPO implemented? Please specify their objectives, policy probability calculations or surrogates, trainable parameters, reward/preference data sources, sampling and update counts, and data or interaction budgets.

  5. How are the confidence head’s ground-truth labels constructed? The paper mentions binary cross-entropy but does not define ground truth targets. Please clarify what c_t measures and how its labels are obtained in simulation and real robot data.

  6. Is the world model robust to unseen distractors? Please compare generated futures with and without distractors against the corresponding real futures. In particular, can it preserve object and scene consistency while accurately predicting target motion, contact interactions and manipulation outcomes?

Reviewer_LBcUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies world-model-guided post-training of vision-language-action models. WISE uses a visual scheduler to select interaction-relevant states, predicts bounded future trajectories for multiple candidate actions, and uses a reward model to compare their outcomes and refine a task-specific SFT policy. Policy updates supervise only the initial action chunk from a real observation context; subsequent imagined trajectories are used to evaluate the candidates.

Critical strengths
  1. The problem is well motivated and practically relevant.

The method addresses both the computational cost of world-model inference and the potential impact of prediction errors on policy learning. Treating when to invoke imagination as a separate design problem is a reasonable approach.

  1. The evaluation covers two VLA backbones, simulation, and real-world tasks.

Reward-component ablations and comparisons with uniform and random scheduling under similar budgets provide useful support for the method’s effectiveness in the evaluated settings.

Critical weaknesses
  1. Comparisons with directly related methods are missing.

The paper discusses WMPO, World4RL, and VLA-RFT but does not compare against them experimentally, leaving WISE’s performance and efficiency relative to existing world-model-guided post-training methods unclear. The scheduling ablations also do not establish the additional benefit of the visual scheduler over simple stage-based rules.

  1. The full training cost and baseline configurations are not sufficiently documented.

Beyond task-specific SFT, WISE requires world-model pretraining and adaptation, as well as scheduler and reward-model training. Table 3 demonstrates savings relative to full imagination, but does not compare the cost of the complete pipeline with conventional post-training approaches. The training configurations, interaction budgets, and tuning budgets of the baselines are also insufficiently specified to verify whether the comparisons use matched conditions.

Questions for authors
  1. What does the confidence target represent, and how are its labels constructed? Figure 7 demonstrates discrimination between successful and failed videos, but have you also evaluated whether the predicted ranking of candidate actions from the same state agrees with their actual execution outcomes?

  2. Since the scheduler’s weak labels are derived from gripper and grasp states, what additional benefit does the visual scheduler provide over directly using these signals or simple stage-based rules when they are available? If these signals are unavailable when scheduling imagination, please clarify that limitation.

Reviewer_pktQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes WISE, an efficient world-model-based post-training framework for Vision-Language-Action (VLA) models. To address the high computational cost of world models and the accumulation of prediction errors in long-horizon rollouts, the authors introduce an Imagination Scheduling mechanism that performs bounded future predictions only at critical interaction stages. The predicted rewards of candidate actions are then used to construct relative advantage signals for VLA policy optimization. Experiments on five MimicGen simulation tasks and four real-world manipulation tasks demonstrate improvements in task success rates while reducing the computational cost of world-model imagination.

Critical strengths
  1. This work explores the integration of Action-Conditioned World Models into VLA post-training. Considering the high computational cost of world-model inference, the authors propose a selective imagination scheduling framework. Experimental results show that the method can significantly reduce world-model computation while improving VLA success rates in both simulation and real-world manipulation tasks.
  2. Experimental results show that, on the π0 backbone, the proposed method achieves a higher average task success rate than post-training baselines such as DPO, GRPO, and DSRL. Furthermore, the authors demonstrate the effectiveness of the method on π0.5 and show performance improvements across different manipulation tasks and generalization settings in real-world experiments.
Critical weaknesses
  1. The method uses weak supervision from Grasp and Active Manipulation states to train a visual classifier, which is relatively straightforward. Comparisons with simple rule-based scheduling using gripper or proprioceptive states are missing.
  2. The paper lacks post-training baselines for π0.5. Further experiments on different VLA architectures, such as SmolVLA, are needed to validate the generality of WISE.
  3. The efficiency evaluation mainly focuses on world-model computation in simulation, while real-world training also involves data collection, environment resets, and rollouts. Comparisons with methods such as RL-Token and DAgger in terms of training costs and performance are needed.
  4. The paper lacks comparisons between predicted and actual outcomes of different actions from the same initial state, especially during contact-rich stages where subtle action differences may lead to different outcomes.
Questions for authors
  1. Regarding training costs, beyond GPU computation alone, what are the actual post-training times and computational resources required by DPO and WISE on the same tasks in the real-world experiments?
  2. The appendix states that task-specific world-model fine-tuning uses 300 demonstrations in simulation and 100 demonstrations in the real world, but the proportions of successful and failed trajectories are not specified. If the model is mainly trained on successful demonstrations, could its ability to predict candidate actions that deviate from the demonstration distribution be affected? Given that predicted videos are directly used to compute action rewards, how do the authors ensure that the model can accurately distinguish the outcomes caused by subtle differences in candidate actions?
  3. Could the authors visualize the predicted outcomes of different candidate actions from the same initial state and compare them with the corresponding real execution results to validate the model's ability to distinguish action consequences?
  4. Can WISE be adapted to VLA backbones with different architectures? Given that the current experiments only cover the π0 family, could the authors further clarify the applicability of the framework to other VLA policies and provide corresponding experimental validation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.