acceptodds
Under review as a conference paper at ICLR 2027

WorldGuide: Learning Success–Failure Boundaries in Latent World Models for Vision-Language-Action Policies

Abstract

Latent world models offer a promising way to improve Vision-Language-Action policies by capturing the consequences of actions. However, models trained primarily on expert demonstrations have limited exposure to failure outcomes and may struggle to distinguish visually similar successful and failed interactions. We propose WorldGuide, a framework that learns these distinctions in latent space and uses them to guide policy training. WorldGuide combines predictive pretraining on successful and failed trajectories with contrastive learning on matched success–failure pairs. The learned predictor then provides a differentiable reward to guide joint optimization of the policy and visual encoder. The predictor is discarded after training, so deployment requires no additional world-model inference. Extensive experiments show that WorldGuide substantially improves VLA reliability and achieves state of the art performance on LIBERO 100 and SimplerEnv, reaching 96.8% and 72.0%, respectively. Code will be publicly available.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotic Manipulation, Latent World Model, Embodied AI, VLA

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QxauOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WorldGuide, a three-stage framework for improving vision-language-action policies with failure-aware latent world modeling. The method first pretrains a latent world model on both successful and failed trajectories, then applies contrastive learning to matched success–failure pairs from the same task and similar execution stages. Finally, the frozen world model provides a differentiable reward for end-to-end VLA policy optimization and is discarded at deployment. Experiments on LIBERO, RoboTwin, SimplerEnv, and a real-robot setup show improved success rates with no additional world-model inference cost.

Critical strengths
  1. Broad empirical evaluation. The method is evaluated across multiple simulation benchmarks and a real-robot setup, with consistent improvements over the reported baselines.

  2. Clear motivation. The paper identifies an important limitation of success-only world-model training: visually similar successful and failed interactions may receive similar latent feedback.

Critical weaknesses
  1. Limited methodological novelty. Stages 1 and 2 mainly use general methods for latent world-model training, such as predictive learning on success/failure data and contrastive separation. World-model-based auxiliary guidance has also been explored in prior works such as VLA-JEPA and VLA-RFT. It is therefore unclear what substantive innovation distinguishes WorldGuide from an engineering combination of existing components.
  2. The contribution of the proposed WM loss is not causally isolated. The paper does not report a controlled (\beta=0) ablation with the same Stage-2 initialization and optimization budget. The default (\beta=0.1) is relatively small, while increasing it to (0.2) causes a clear performance drop, suggesting that BC remains the main learning signal. Thus, the actual contribution of the WM-guided loss is unclear.
  3. The WM reward is an insufficiently validated proxy. The reward is based on latent distance to a successful future rather than true task success. The paper does not test reward calibration, candidate-action ranking against real outcomes, or possible exploitation of errors in the frozen world model. Hence, the observed gains do not establish that this reward is reliably aligned with task completion.
  4. Limited generality of the failure-data pipeline. Failure data require simulator perturbations, under-converged policy checkpoints, VLM-based failure-onset annotation, and manual verification. It remains unclear whether the method works with naturally collected failures, noisy annotations, or without this additional supervision.
  5. Insufficient real-world evidence. The real-robot evaluation uses only three tasks and 20 trials per task. The results are therefore suggestive but insufficient to support strong claims about robustness and generalization.
Questions for authors

see weakness above

3 comments
AuthorsOpenReview ↗

Response to Reviewer Qxau (1/2)

Thank you for your detailed feedback. The modified parts in the revised PDF are marked in blue.

W1. Clarification on Novelty
While WorldGuide uses world models to improve VLA learning, its objective and learning mechanism differ fundamentally from prior work.

Specifically, VLA-JEPA focuses on learning action-relevant representations through leakage-free future latent prediction, jointly optimizing representation learning and action generation. VLA-RFT uses an autoregressive world model as a visual simulator, generating policy rollouts and optimizing the VLA through verified visual rewards and GRPO.

Instead of relying on future prediction similarity, WorldGuide explicitly learns outcome-sensitive success/failure boundaries through three key designs: (1) a failure-rich dataset covering both coarse and subtle execution failures. (2) progress-anchored, nearest-neighbor contrastive learning with spatiotemporal weighting to isolate outcome-critical deviations. And (3) a differentiable latent boundary reward that provides direct action-level gradients for policy optimization, requiring neither autoregressive rollouts nor extra inference overhead. Our ablations further support this distinction: the proposed boundary reward achieves an AUC of 0.85, compared with 0.47–0.50 for prediction MSE and goal similarity.

W2. Contribution of the World-Model Reward under Controlled β\beta Ablations
Setting β\beta=0 removes the world-model reward term, reducing the objective to BC-only baseline. Our results show that introducing the boundary-aware reward β\beta=0.1 improves over this baseline. Although increasing β\beta to 0.2 reduces performance by 3.3 percentage points compared with β\beta=0.1, it still outperforms the BC-only baseline. This indicates that the learned reward provides useful policy guidance, while excessive reward weighting compromises the balance between boundary-aware optimization and behavioral imitation. Thus, the performance drop at β\beta=0.2 reflects an imbalance between the two objectives rather than a lack of contribution from the world model. The corresponding results are summarized below:

Reward weight (β)Robotwin (avg)
0.080.1
0.185.9
0.282.6

W3. Local Reward Reliability
To examine whether the learned reward aligns with actual execution outcomes, we conducted local action perturbation experiments on LIBERO, fixing the 9-frame visual context and applying Gaussian noise to successful action chunks.

Perturbation scaleRanking accuracy95% CInSuccess-rate drop
σ = 0.050.694[0.619, 0.769]1601.3%
σ = 0.100.731[0.656, 0.806]1609.5%
σ = 0.150.856[0.788, 0.919]16014.7%

Ranking accuracy measures how often the reward assigns a higher score to the successful action than to its perturbed counterpart (chance = 0.5). The increasing ranking accuracy, together with the corresponding decline in execution success, suggests that the learned reward captures meaningful local differences in action quality.

W4. Failure Data Generalization and Annotation Robustness
Our failure data are not limited to simulator perturbations. We also execute under-converged policy checkpoints in the simulator and collect the resulting failures, allowing us to continuously expand the failure dataset. Regarding annotation, VLM-generated failure timestamps are used only for temporal matching in Stage 2, rather than as direct supervision for world-model prediction or reward learning. Stage 1 can also learn from failure trajectories without precise failure timestamps.

We also introduced a 0.2s perturbation to failure-onset timestamps. Although performance decreased slightly, it remained above the baseline, indicating robustness to moderate annotation noise.

BenchmarkSuccess Rate (%)
RoboTwin84.5
LIBERO (Average)97.8
AuthorsOpenReview ↗

Response to Reviewer Qxau (2/2)

Thank you for your detailed feedback. The modified parts in the revised PDF are marked in blue.

W4. Insufficient Real-World Evidence

We further expanded our real-world evaluation by introducing three additional manipulation tasks: orange oval, green star, and blue trapezoid. We also increased the number of trials from 20 to 60 per task for each method. The expanded results show consistent improvements with WorldGuide across a broader range of object geometries, providing stronger evidence of its effectiveness and robustness in real-world manipulation. Additional experiments can be found in Appendix E.

Taskπ0 (95% CI)π0.5 (95% CI)WorldGuide (95% CI)
Purple square11.7% [5.8, 22.2]10.0% [4.7, 20.1]23.3% [14.4, 35.4]
Green circle20.0% [11.8, 31.8]15.0% [8.1, 26.1]20.0% [11.8, 31.8]
Yellow triangle5.0% [1.7, 13.7]10.0% [4.7, 20.1]16.7% [9.3, 28.0]
Orange oval*16.7% [9.3, 28.0]15.0% [8.1, 26.1]25.0% [15.8, 37.2]
Green star*6.7% [2.6, 15.9]5.0% [1.7, 13.7]11.7% [5.8, 22.2]
Blue trapezoid*13.3% [6.9, 24.2]11.7% [5.8, 22.2]21.7% [13.1, 33.6]
Overall*12.2% [9.2, 16.0]11.1% [8.3, 14.8]19.7% [15.9, 24.1]
AuthorsOpenReview ↗

Kind Request for Further Feedback

Dear Reviewer Qxau,

As the discussion period is nearing its end, we would appreciate any feedback on whether our responses and additional experiments have addressed your concerns. If any questions remain, we would be happy to provide further clarification. Thank you again for your time and thoughtful feedback.

Reviewer_W9nnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses failure-aware latent world models to improve vision-language-action policies without additional inference-time computation. The method is three-fold: first learns predictive representations from successful and failed trajectories, then separates visually matched success–failure segments through contrastive fine-tuning. Finally, a frozen predictor provides a differentiable goal-distance reward alongside behavior cloning. Experiments evaluate policy performance across LIBERO, SimplerEnv, selected RoboTwin tasks, and real-world manipulation.

Critical strengths

The paper addresses an important limitation of demonstration-based predictive learning: visually small execution errors can have large consequences for task success. Matching successful and failed segments before contrastive training is a sensible way to reduce irrelevant differences between examples. The stage-wise and pairing ablations provide useful support for the recipe, with consistent policy improvements across simulation benchmarks. The episode-held-out probes also offer evidence that contrastive fine-tuning improves outcome discrimination. Discarding the world model after training is a practical advantage.

Critical weaknesses

The central question is whether better outcome discrimination produces better action-improvement gradients. The contrastive objective separates successful and failed predictions, but this does not necessarily make the resulting distance locally useful for optimizing actions. Likewise, the linear-probe results demonstrate separability rather than the correctness of reward gradients for policy-proposed actions. The reward-source ablation supports the overall recipe, but more direct evidence linking changes in predicted reward to changes in actual execution success would strengthen the proposed explanation.

The construction of the Stage-3 reward also needs clarification. The appendix describes next-frame prediction under causal teacher forcing, while Equation 8 evaluates a policy-generated action chunk against a successful future. It is unclear which intermediate observations are supplied when evaluating that chunk and how successful reference windows are aligned with rollout states. A demonstration from the same task may differ in scene configuration or task progress, making its latent distance an imperfect measure of action quality. The real-world evaluation provides limited additional support: success increases from 7/60 to 11/60 trials, but no matched base-policy ablation or uncertainty estimates are reported.

Questions for authors
  1. Is existing evidence available that increasing the learned reward through action updates improves actual execution outcomes? In particular, how reliable is the reward under local perturbations of policy-proposed actions?

  2. How is a complete policy-generated action chunk evaluated by the teacher-forced predictor in Stage 3? Please clarify which observations are supplied and whether future latents are rolled out autoregressively.

  3. How are successful reference windows selected and aligned with policy-rollout states, especially when scene configuration or task progress differs?

  4. Please provide available uncertainty estimates and a matched base-policy comparison for the real-world results.

3 comments
AuthorsOpenReview ↗

Response to Reviewer W9nn

Thank you for your detailed feedback. The modified parts in the revised PDF are marked in blue.

W1 & Q1. Local Reward Reliability and Its Alignment with Actual Execution Success
To evaluate whether the learned reward possesses sufficient discriminative capability to yield more effective gradients for action optimization, we conducted tests on LIBERO. With a fixed 9-frame visual context, we applied Gaussian noise perturbations to the successful actions, and the results are as follows:

Perturbation scaleRanking accuracy95% CInSuccess-rate drop
σ = 0.050.694[0.619, 0.769]1601.3%
σ = 0.100.731[0.656, 0.806]1609.5%
σ = 0.150.856[0.788, 0.919]16014.7%

Ranking accuracy measures the probability that the world model assigns a higher reward to the successful action than to the noisy action. The results show that as the perturbation magnitude increases, the reward increasingly prefers the demonstrated action over the perturbed alternatives. This indicates that the reward learned by WorldGuide provides a meaningful local ranking signal. The corresponding drop in success rate further illustrates the performance differences associated with the evaluated settings.

W2 & Q2. Stage 3: Reward Construction
We clarify that Stage 3 follows the teacher-forced prediction paradigm, using only successful expert trajectories. Specifically, we sample a state (sts_t) and its corresponding future window from the same successful expert trajectory. The policy takes (sts_t) as input and generates a complete action chunk (a^t:t+W−1\hat{a}_{t:t+W-1}). To evaluate this chunk, the world model uses the ground-truth expert observations as causal contexts and the corresponding policy-generated actions as inputs. At each temporal position, it predicts the next-frame latent and compares it with the encoded ground-truth future frame from the same trajectory.

Importantly, the predictions at all temporal positions are computed in parallel using a causal attention mask. Ground-truth intermediate observations are used for teacher forcing; predicted latents are not fed back into subsequent steps. Therefore, Stage 3 evaluates the entire action chunk through parallel one-step predictions rather than autoregressive rollout.​ Since the reference future window comes from the same expert trajectory as the sampled initial state, the scene configuration and task progress are naturally aligned, avoiding mismatches introduced by randomly pairing demonstrations from the same task. We clarify these details in the manuscript Appendix A.

Q3. Successful Reference Selection and Temporal Alignment Across Scene Configurations
Our wording “policy rollouts” in Section 3.4 was imprecise: Stage 3 uses policy-conditioned future predictions from expert observations, rather than independently collected online rollout states requiring reference retrieval.
 Each training sample contains a current observation �0��0� and its temporally aligned GT future observation �1��1�, both taken from the same successful expert trajectory. The policy receives �2��2� and the task instruction and predicts an action chunk �3��3�. The frozen world model then predicts the future latent representation conditioned on this observation and the predicted actions. The corresponding GT observation is encoded by the frozen world-model encoder to provide the target representation. The predicted and reference future representations are defined as:

z^t:t+W=gψ(ϕ(st),a^t:t+W)\hat{z}_{t:t+W} = g_{\psi}(\phi(s_t), \hat{a}_{t:t+W}) zt:t+W=ϕ(st:t+W)z_{t:t+W} = \phi(s_{t:t+W})
The reward is: rt=−dm(z^t:t+W,zt:t+W)r_t = -d_m(\hat{z}_{t:t+W}, z_{t:t+W})
Thus, the reference is determined directly by the sampled expert trajectory and temporal window; it is neither randomly selected nor retrieved from another successful execution.
Q4. Uncertainty Estimation on Physical Robots
We expanded our real-world evaluation with two additional rounds (20 trials per task and method each) under identical task configurations and evaluation protocols, yielding 60 trials per task (180 per method) across all three rounds. Using 95% Wilson confidence intervals to quantify uncertainty, WorldGuide achieves an overall success rate of 20.0% versus 12.2% for π0\pi_0 and 11.7% for π0.5\pi_{0.5}—relative gains of 63.6% and 71.4%, respectively:

Taskπ0 (95% CI)π0.5 (95% CI)WorldGuide (95% CI)
Purple square11.7% [5.8, 22.2]10.0% [4.7, 20.1]23.3% [14.4, 35.4]
Green circle20.0% [11.8, 31.8]15.0% [8.1, 26.1]20.0% [11.8, 31.8]
Yellow triangle5.0% [1.7, 13.7]10.0% [4.7, 20.1]16.7% [9.3, 28.0]
Overall12.2% [8.2, 17.8]11.7% [7.8, 17.2]20.0% [14.8, 26.4]

We have included the expanded results and uncertainty estimates in the revised appendix E.

Thank you for the response and additional experiments. That said, my main concerns remain unresolved. First, Stage 3 relies on teacher-forced single-step predictions rather than multi-step rollouts, which is essentially a perceptual behavior cloning disconnected from Stage 2's failure representations. Second, the perturbation study does not sufficiently support the claim. Since the paper use the prediction error for reward at stage 3, without comparing with stage 1 ranking accuracy, the acclaimed numbers cannot support the claim. Also the Gaussian noise is fall from the policy error during training, which makes the experimental setup weak. Lastly, the real-world evaluation still omits the requested matched ablation (e.g., training non-contrastive stage 1 model). Also the success rate remains very low (e.g., 20%), and it overlaps with other policy models, showing no strict gains in terms of confidence intervals.

Accordingly, I will maintain my current score.

AuthorsOpenReview ↗

Response to Reviewer W9nn

Thank you for the response.

Regarding the concern that teacher-forced prediction might disconnect Stage 3 from Stage 2, we clarify two key points: First, teacher-forced causal prediction is a standard design also used in VLA-JEPA to jointly optimize future latent prediction and action generation. Using teacher forcing does not inherently separate world-model learning from policy optimization. More importantly, Stage 3 directly builds upon the Stage-2 fine-tuned world model, rather than the Stage-1 predictor. The failure-aware representations and the spatiotemporally weighted distance (dmd_{m}) learned during Stage 2 are preserved and directly define the differentiable reward. Even though predictions are computed in parallel instead of autoregressively, the reward gradients still flow back through the frozen world model to update both the action head and the visual encoder.

Regarding the validity of our perturbation analysis and the contribution of Stage 2, we conducted additional experiments to evaluate the reliability of the learned reward. Specifically, we directly compared the reward ranking accuracy between failed and successful action trajectories generated by the model (results shown below). The results demonstrate that Stage 2 yields more discriminative rewards with superior ranking capability.

Stage 1Stage 2Improvement
0.6680.787+11.9 pp

Regarding the concerns about real-world evaluations, we apologize for omitting the corresponding ablation study in our previous response and now report the results in the table below. These results clearly demonstrate the consistent performance gains achieved by Stage 2 in physical robot deployments.

TaskStage 1Stage 2
Purple square13.3%23.3%
Green circle16.7%20.0%
Yellow triangle10.0%16.7%
Orange oval*20.0%25.0%
Green star*11.7%11.7%
Blue trapezoid*15.0%21.7%
Overall*14.4%19.7%

Furthermore, regarding the low absolute success rates, we would like to clarify that our experiments involve a wooden shape-matching board conducted on the ARX Lift 2 platform. This is an inherently intricate, contact-rich fine-manipulation task, where even state-of-the-art models like π0.5\pi_{0.5} struggle. We deliberately chose this challenging setup to highlight that, under fine-grained tasks, successful and failed trajectories must be sharply differentiated by the world model to provide reliable training signals for the VLA policy.

Reviewer_eKfxOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

WorldGuide improves vision-language-action policies through failure-aware latent world modeling. The method first learns predictive dynamics from successful and failed trajectories, then applies contrastive learning to matched success-failure segments from the same task. The resulting world model provides a differentiable reward for policy optimization and is discarded at deployment.

The approach is evaluated on LIBERO, a 12-task subset of RoboTwin, SimplerEnv, and three real-robot manipulation tasks. Stage-wise ablations, reward comparisons, and representation analyses investigate the contributions of failure exposure and boundary-aware supervision.

Critical strengths
  1. A clear motivation and a practical use of failure data. The authors use failed executions to broaden world-model training, then compare similar successful and failed segments from the same task. This helps emphasize small differences in grasping or alignment that can determine task completion. The trained world model then guides policy training and is discarded at deployment. This provides a concrete way to use failure experience to improve the policy without adding world-model inference during execution.

  2. Targeted ablations support the main design choices. Table 5 shows progressive improvements from success-only world-model training, adding failure data, and adding boundary-aware contrastive learning. Table 8 further examines how the samples are paired: RoboTwin clean success rises from 78.3% with random pairing to 84.7% with progress matching and 85.7% with the full matching method. This shows that comparing executions at similar stages matters. The analysis in Table 4 also supports improved discrimination of near-failure segments. Together, these experiments help explain which parts of the method contribute to the gains.

Critical weaknesses
  1. The comparison settings need clearer documentation. Averaging the public StarVLA-OFT results over the corresponding 12 task names gives 90.33%/92.25% in the official Easy/Hard columns [1], versus the paper's reported clean/randomized baseline of 80.1%/80.2%. The public model jointly trains all 50 tasks, so the gap alone does not establish an unfair comparison. Clarifying the baseline recipe and rollout-data budget would help interpret the gains. Figure 5 should also clarify whether the reward variants share a frozen world-model checkpoint, isolating reward readout from representation learning.

  2. Robustness to unseen changes and reward behavior need clearer characterization. The randomized RoboTwin and SimplerEnv evaluations provide encouraging evidence, but the relationship between training perturbations and evaluation shifts is not sufficiently specified. A focused held-out perturbation evaluation would strengthen the broader robustness claim. In addition, Section 3.4 does not fully specify how successful reference segments are selected for rollout states when scene configuration or execution progress differs. This affects the interpretation of the reward for different but valid execution paths.

[1] Official StarVLA results and training setup. The 12-task averages are computed from the corresponding rows of the 50-task co-trained model.

Questions for authors
  1. RoboTwin baseline settings. Please clarify the OpenVLA-OFT/StarVLA-OFT naming and whether successful demonstrations, VLA initialization, policy-training budget, checkpoint selection, and evaluation settings were matched. Which differences might explain the gap to public results? Were Stage-3 rollout states collected beforehand or through additional interaction, and what was their quantity?

  2. Stage-3 successful references. How are successful reference segments selected for policy-rollout states? Are they matched by task progress or scene configuration? Please explain how this reward accommodates different but valid execution paths.

  3. Figure 5 reward comparison. Do all reward variants use the same frozen world-model checkpoint? Please clarify the prediction-MSE baseline, particularly the reference to a success-only world model. If the checkpoint is already fixed, an explicit statement would resolve this concern.

  4. Robustness to unseen perturbations. Which evaluation perturbations are absent from training? A matched comparison on one suitable setting, such as LIBERO-Plus [2], LIBERO-PRO [3], or RoboTwin clean-to-randomized (C2R) [4], would strengthen the claim. For C2R, please disclose exposure to randomized conditions across world-model training, failure-data collection, and policy training, together with any additional data or interaction budget.

[2] LIBERO-Plus;

[3] LIBERO-PRO;

[4] RoboTwin official C2R protocol.

4 comments
AuthorsOpenReview ↗

Response to Reviewer eKfx (1/2)

Thank you for your detailed feedback. The modified parts in the revised PDF are marked in blue.

W1. Unclear baseline training settings and reward ablation design

The discrepancy stems from training scale differences: the released StarVLA-OFT checkpoint was jointly trained on all 50 RoboTwin tasks for 140K steps on 48 A100 GPUs, whereas our baseline was trained on 12 selected tasks for 110K steps on 8 H20 GPUs. Moreover, community reports in the official repository confirm substantial reproduction gaps even under identical configurations, motivating our use of self-reproduced baseline results.

In Fig.5, the variants use different world-model checkpoints: Prediction MSE and Goal Similarity use a model trained only on successful trajectories, while Boundary (ours) uses the failure-aware model trained through Stages 1 and 2. Fig.5 complements the stage-wise ablation in Tab.5, rather than serving as an isolated ablation of reward functions under a fixed representation.

W2. Unseen-Perturbation Robustness and Successful Reference Segment Selection
Robustness under distribution shifts. We additionally evaluate on LIBERO-Plus, which covers seven perturbation categories. Success rates (%) are reported below:

ModelCameraRobotLanguageLightBackgroundNoiseLayoutTotal
StarVLA-OFT47.060.187.096.395.373.179.275.0
VLA-JEPA40.355.772.988.270.538.274.662.9
WorldGuide48.161.385.997.395.280.381.376.8

WorldGuide achieves 76.8% overall success, exceeding the listed StarVLA-OFT result by 1.8 percentage points. These results provide additional evidence of robustness across distribution shifts, without implying uniform gains across all perturbations.

Successful-reference selection in Section 3.4. Our wording “policy rollouts” in Section 3.4 was imprecise: Stage 3 uses policy-conditioned future predictions from expert observations, rather than independently collected online rollout states requiring reference retrieval.

How the successful reference is selected. Each training sample contains a current observation �0��0� and its temporally aligned GT future observation �1��1�, both taken from the same successful expert trajectory. The policy receives �2��2� and the task instruction and predicts an action chunk �3��3�. The frozen world model then predicts the future latent representation conditioned on this observation and the predicted actions. The corresponding GT observation is encoded by the frozen world-model encoder to provide the target representation. The predicted and reference future representations are defined as:

z^t:t+W=gψ(ϕ(st),a^t:t+W)\hat{z}_{t:t+W} = g_{\psi}(\phi(s_t), \hat{a}_{t:t+W}) zt:t+W=ϕ(st:t+W)z_{t:t+W} = \phi(s_{t:t+W})

The reward is: rt=−dm(z^t:t+W,zt:t+W)r_t = -d_m(\hat{z}_{t:t+W}, z_{t:t+W})
Thus, the reference is determined directly by the sampled expert trajectory and temporal window; it is neither randomly selected nor retrieved from another successful execution.
How the reward treats alternative valid paths. The reward compares future latent representations, not action sequences. Consequently, different action chunks can receive similar rewards if they produce similar representations of task-relevant future states. The temporal and channel weighting in �4��4� is intended to emphasize outcome-relevant differences and reduce the influence of incidental variation. However, this does not make the reward invariant to all successful paths: a valid continuation with substantially different timing, intermediate states, or latent features may receive a lower reward than the demonstrated continuation. We therefore interpret it as a reference-conditioned auxiliary training signal, combined with the behavioral-cloning objective, rather than a universal success criterion for arbitrary trajectories. The world model and reference video are used only during training and are not required at deployment.

AuthorsOpenReview ↗

Response to Reviewer eKfx (2/2)

Q1. RoboTwin baseline settings.

In Section 4.1, we mistakenly referred to StarVLA-OFT as OpenVLA-OFT. This error has been corrected. The discrepancy between our baseline and the publicly reported results mainly stems from differences in training settings. StarVLA-OFT was trained on all 50 tasks for 140K steps using 48 A100 GPUs (per-device batch size of 4), whereas our baseline was trained on only 12 tasks for 110K steps using 8 H20 GPUs with the same per-device batch size. We have also clarified the Stage 3 description in our response to the corresponding weakness 1. Our training data consists of 6.6K trajectories from the 12 selected tasks in the official dataset.

Q2. Stage-3 successful references.
We provide a more detailed explanation in our response to Weakness 2. Successful reference segments are not selected through task-progress estimation or scene-configuration matching. Instead, the current observation and reference future are drawn from the same recorded expert trajectory, ensuring their temporal alignment and scene consistency. No additional cross-trajectory matching is required. Although the predicted actions may produce a different future from the expert continuation, both are evaluated from the same initial observed state.

Q3. Figure 5 reward comparison
We clarify this point in our response to Weakness 1. In Fig. 5, the variants use different world-model checkpoints: Prediction MSE and Goal Similarity use a model trained only on successful trajectories, whereas Boundary uses a model trained through Stages 1 and 2. This is because our distance function (d_m) is essentially MSE-based (Eq. 8). The purpose of Fig. 5 is to investigate how incorporating failure data improves world-model representations, rather than to compare different reward formulations.

Q4. Robustness to unseen perturbations.
In our response to Weakness 1, we provide additional results on LIBERO-Plus, demonstrating that our method generalizes across diverse perturbation categories. For C2R, our policy is not exposed to randomized conditions during training. The successful and failed trajectories collected from RoboTwin evaluations are used exclusively for world-model training, while Stage 3 policy training relies solely on the official training dataset. The training settings are detailed in Appendix A.2 and Section 4.1.

Thank you for the detailed responses and the additional LIBERO-Plus evaluation. The two responses to my review address my main questions and strengthen my support for acceptance. The clarification of the RoboTwin training scale and the correction of the baseline name provide a reasonable explanation for the gap to the public StarVLA-OFT results. I do not regard that gap alone as evidence of an unfair comparison. My concern about Stage-3 reference selection is resolved. The current observation and target future come from the same successful expert trajectory, with temporal alignment and a consistent scene. This removes the cross-trajectory matching concern raised in my review. I also appreciate the explicit acknowledgment that the reward is a reference-conditioned auxiliary signal used alongside behavioral cloning, rather than a universal success criterion for all valid continuations. For Figure 5, the confirmation that the variants use different world-model checkpoints answers my question about the experimental setup. The figure should therefore be interpreted as comparing combinations of representation learning and reward construction, rather than isolating the reward formulation under a fixed representation. Please reflect this distinction in the revised text and caption. The LIBERO-Plus results directly address my request for an additional robustness evaluation. The reported overall success rates of 76.8% for WorldGuide and 75.0% for the listed StarVLA-OFT baseline provide positive additional evidence. The authors appropriately acknowledge that the gains are not uniform across perturbation categories. I do not require a separate LIBERO-PRO or C2R experiment to address this request. Overall, I find the responses satisfactory on the main points raised in my review. I continue to value the practical use of failure data and the targeted ablations, and I maintain my rating of 3 (weak acceptance), with increased confidence in this recommendation. Please incorporate these clarifications into the revised manuscript.

AuthorsOpenReview ↗

Thank You for Your Positive Assessment of Our Work

Thank You for Your Positive Assessment of Our Work

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.