acceptodds
Under review as a conference paper at ICLR 2027

Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning

Abstract

Action-conditioned world models increasingly serve as learned simulators for robotic policy improvement, yet their reliability rests on an unverified assumption: generated futures faithfully reflect the consequences of arbitrary valid actions. Systematic evaluation against action-specific ground truth remains limited beyond expert demonstrations. To address this gap, we introduce WorldEcho, which evaluates action following over a broader action distribution through visual integrity and end-effector trajectory alignment. Our diagnosis shows that expert-trained models perform better on demonstrated actions but struggle with off-expert trajectories, producing motion inconsistent with the commanded actions or visually invalid rollouts. To mitigate these failures, we propose WorldSync, which broadens action coverage, grounds intermediate video representations in action-induced robot dynamics through an Action-Forcing Expert, and aligns predicted and ground-truth changes under action interventions. Experiments show that WorldSync reduces integrity-gated error on RoboTwin and, when used for iterative policy improvement, yields higher success rates in simulation and on real robots. Project page: https://worldecho-worldsync.github.io

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, action-conditioned video generation, robot manipulation, benchmark, policy learning

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_EiuEOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces WorldEcho, a benchmark that evaluates whether action-conditioned robotic world models generate futures consistent with the specified actions. It compares generated videos with matched simulator rollouts across expert and several off-expert action queries, measuring both visual integrity and end-effector trajectory alignment. Motivated by the observed off-expert failures, the authors propose WorldSync, which combines expanded action coverage with Action-Forcing Expert and Intervention-Effect supervision. The evaluation covers 50 RoboTwin tasks, multiple world-model baselines, and downstream policy-improvement experiments in simulation and on real robots.

Critical strengths
  • A well-motivated and substantial evaluation. WorldEcho tests action following against action-specific simulator outcomes, rather than relying solely on visual similarity. The range of action queries, baselines, and reported metrics makes the observed failures informative.

  • A coherent connection from diagnosis to intervention and downstream validation. The authors use the identified failures to motivate concrete training changes, provide model-level ablations, and examine whether the resulting model benefits policy learning. The combination of world-model and policy experiments is a strength of the paper.

Critical weaknesses

I find this paper solid. I have no major concerns, but I have a few suggestions that could deepen the analysis.

Among the three improvements, expanding the training data distribution appears to be the most important. The importance of this is well established in model-based RL, as discussed in Agnostic System Identification for Model-Based Reinforcement Learning. Subsequent MBRL research could also inform the paper’s policy-learning analysis. A model need not be accurate everywhere; accuracy near the distribution visited by the policy may matter more (Live in the Moment: Learning Dynamics Model Adapted to Evolving Policy). When the model is unreliable, the length of model-generated rollouts can also be limited, as in MBPO (When to Trust Your Model: Model-Based Policy Optimization). Finally, the policy-learning experiments use only the full WorldSync model, without examining how each proposed improvement affects policy performance. Component-level policy ablations might reveal which improvements matter most and motivate a deeper analysis of model utility.

Questions for authors

Would the authors be able to report a downstream policy-improvement ablation using the same policy-training pipeline with expanded-data Base, Base + IE, Base + AFE, and full WorldSync? Even a compact experiment on the simulation task would help determine whether the components that improve WorldEcho scores also improve policy learning.

3 comments
AuthorsOpenReview ↗

Response to Reviewer EiuE

Thank you for the supportive review and the thoughtful suggestions. We are encouraged that you "find this paper solid" with "no major concerns". We have run the ablation you asked for and added the discussion you suggested.

Weakness and Question: Component-level downstream policy ablation

Short answer. Yes. We ran exactly these four configurations under the same policy-training pipeline, extended to three rounds. Each component improves policy learning over expanded-data Base in every round. After three rounds the combination improves it most, and the order of the four matches the order of their WorldEcho gated errors.

World model (expanded data)Gated error ↓Initial (%)Round 1 (%) ↑Round 2 (%) ↑Round 3 (%) ↑
Base0.026452555863
Base + IE0.021452606367
Base + AFE0.023252576266
WorldSync (IE + AFE)0.019352576570

Gated errors are those of Table 3. Expert Base and both CtrlWorld checkpoints are in Rebuttal Table 1 of the General Response (2/2).

  • The components that lower WorldEcho error also improve policy learning. After Rounds 2 and 3, the order by gated error is the order by success rate. The match is strong but not exact: after Round 1, Base + IE is at 60% and full WorldSync at 57%.
  • IE and AFE contribute similarly, and their final effects add. After three rounds IE adds 4 points over Base, AFE adds 3, and the two together add 7.
  • Coverage is central, as you note. Expanded data accounts for 7 of the 14 points between Expert Base (56%) and WorldSync (70%), and the two objectives for the other 7. In Round 3 both expert-trained models stop improving (Expert Base 58% to 56%, CtrlWorld 57% to 56%), while every expanded-data model continues to improve.

Revision. Section 4.3 (Table 2) and Appendix C.3 (Table 8) now include these conditions.


Suggestion: Connections to model-based reinforcement learning

Thank you for these pointers, which place our findings in a broader context. We now cite the three works, with a short discussion in Sections 2.2 and 5:

  • Data coverage (Ross & Bagnell, 2012). Our results agree with this work. Expanded coverage lowers gated error for all six baselines (Table 1) and raises policy success by 5 to 7 points after three rounds. Policy Rollout data gives the largest single reduction in the data ablation (Table 13).
  • Accuracy near the policy's distribution (Wang et al., 2023). WorldEcho's Policy Rollout category measures action following on policy-generated actions (Appendix F.1), and our pipeline adapts the world model in every round on the current policy's rollouts (Appendix C.2).
  • Rollout length (Janner et al., 2019). Our pipeline uses fixed-horizon rollouts with filtering after generation. Horizon-aware truncation is a natural extension, which we now note as future work in Section 5.

We are happy to discuss any further questions.

Thank you for your reply. I’ll keep my rating.

Thank you for your reply and for your support of the paper. We appreciate the suggestions, which helped us strengthen the paper.

Reviewer_p5K5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that due to a limitation on the datasets currently used in both training and evaluating action-conditioned world models being mostly composed of expert demonstrations, current evaluation methods miss the faliures of these world models exactly where its training dataset is weakest, and thus remain oblivious to non-expert action induced inaccuracies. The paper tackles this limitation in two parts. First, for evaluation, it generates plausible action distributions that are either task adjacent, or simply feasible. It uses a prebuilt physics simulator in order to generate ground truth sequences to compare to model predictions, and apply an integrity gate to raw trajectory alignment ratings. The second part tackles how to mitigate these action dependent failures of existing models. Specifically, it shows that this non-expert data can be used to produce 2 additional supervision signals beyond the default flow matching signal to take full advantage of them, and achieve an overall higher gated score.

Critical strengths

The paper's main goal of demonstrating how current evaluation methods that overly rely on readily available expert demonstration datasets to validate a model's action following creates a gap between the model's perceived capabilities and its true capabilities in the eventual inference use case, which is typically policy evaluation or training. As a benchmark, generating new counter-factual data through a crafted simulator is fully reasonable and results in figure 5 is convincing on the existance of this gap. It then connects this through policy improvement to show that this score does correlate to real increase in training signal when using the world model as a training enviroment.

Critical weaknesses
  • For the policy improvement section's claim that strong WorldEcho performance is connected to larger downstream performance, I feel that this claim is made substantially weaker as the ctrlworld baseline was trained with expert data only, and in my opinion this claim cannot be made without also evaluating the expanded regime crtlworld checkpoint, which also has a closer gap between gated error scores copared to WorldSync. This also weakens WorldSync's claim on both IE and AFE being actually beneficial for downstream use cases. Overall, the lack of datapoints across a larger variety of models and gated/raw errors, across both simulation and real robots, weakens the presented utility of the metric.
  • The use of only NDTW feels not fully justified for a metric that intends to show future policy improvement performance. I see no information that points to a frame aligned metric that should still be provided as an additional signal for velocity of predicted sequences, which can be critical to policy performance in certain tasks.
Questions for authors
  • How were the image-quality threshold, motion-smoothness threshold, and visual-failure penalty selected? And are the leading Table 1 rankings stable under nearby choices?
6 comments
AuthorsOpenReview ↗

Response to Reviewer p5K5 (1/2)

Thank you for the careful and rigorous review, and for finding the simulator-based benchmark "fully reasonable" and the Figure 5 results "convincing". Your comments on the downstream claim and the metric design are valuable. We address them with new experiments in two parts: Weakness 1 (this comment), then Weakness 2 and your question (2/2).

Weakness 1: Evidence linking WorldEcho performance to downstream policy improvement

Thank you for this suggestion. We added the expanded-data CtrlWorld checkpoint and a component ablation: seven world models under the protocol of Section 4.3 (same initial policy at 52%, budgets, and 100 evaluation seeds), extended to three rounds. Per-round success rates are in Rebuttal Table 1 of the General Response (2/2).

(a) Expanded-data CtrlWorld baseline. Expanded data lifts CtrlWorld from 56% to 61% after three rounds, and WorldSync (70%) remains 9 points above this stronger baseline. This result also shows that Expanded Action Coverage is not tied to our architecture: the same data gives an existing world model a moderate downstream gain, in line with Table 1, where it lowers the gated error of all six baselines.

(b) Downstream benefit of IE and AFE. With the data held fixed, expanded-data Base reaches 63% after three rounds, Base + IE 67%, Base + AFE 66%, and full WorldSync 70%. Every variant with IE or AFE is above Base after every round. WorldSync is the best variant after Rounds 2 and 3; after Round 1, Base + IE is ahead (60% vs. 57%). IE and AFE give comparable gains, and their combination gives the largest.

(c) More data points relating the metric to downstream gains. The simulation experiment now covers seven world models instead of two.

Rebuttal Table 2. WorldEcho scores and simulation policy gains in percentage points (Bin Dumping, initial policy to Round 3).

GroupWorld modelDataWorldEcho setGated error ↓Raw NDTW ↓Simulation gain ↑
BaselineCtrlWorldExpertTable 10.03790.0266+4
CtrlWorldExpandedTable 10.02930.0210+9
AblationExpert BaseExpertTable 30.03520.0314+4
BaseExpandedTable 30.02640.0204+11
Base + IEExpandedTable 30.02140.0176+15
Base + AFEExpandedTable 30.02320.0172+14
WorldSync (IE + AFE)ExpandedTable 30.01930.0138+18
  • After three rounds, a lower gated error goes with a larger gain in both groups, without exception. The two groups are scored on different WorldEcho sets, so we compare within each group and do not pool them.
  • The relation is strong but not exact. After Round 2, Expert Base and Base are tied at 58% despite gated errors of 0.0352 and 0.0264. After Round 1, Base + IE (0.0214) is at 60% and WorldSync (0.0193) at 57%. Across the five models of the ablation group, the Spearman correlation between gated error and success rate is −0.82, −0.97, and −1.00 after Rounds 1, 2, and 3; for raw NDTW it is −0.67, −0.87, and −0.90. Gated error indicates which world model is likely to give the larger gain, but it does not replace the policy experiment.
  • The real-robot results of the submission point the same way. On all four tasks of Figure 6, WorldSync, which has the lower gated error, gives the larger gain: +16 to +20 points, versus −8 to +8 for expert-trained CtrlWorld.

Revision. Section 4.3 (Table 2) and Appendix C.3 (Table 8) now report all seven conditions over three rounds, and Rebuttal Table 2 corresponds to Table 9. The closing sentence of Section 4.3 now reads:

Together with the simulation results, these experiments associate stronger WorldEcho performance with larger downstream policy gains. This association is strong but not exact and is limited to the evaluated protocol.

AuthorsOpenReview ↗

Response to Reviewer p5K5 (2/2)

Weakness 2: NDTW alone and the need for a frame-aligned signal

Short answer. NDTW is temporally elastic by design: it measures whether the generated end effector follows the commanded path in the correct order, separately from its timing. Speed is a useful complementary signal, as you point out, so we now report a frame-aligned velocity error, E_vel, alongside NDTW. The two metrics rank the 14 configurations of Table 1 consistently (Spearman ρ = 0.92).

  • NDTW separates path error from timing error. A frame-aligned pose error charges a small temporal lag again at every later frame, so one early delay can dominate the score of a rollout whose path is otherwise correct. NDTW is also the trajectory measure of EWMBench and WorldArena, and it correlates with human ratings of action agreement (Spearman ρ = −0.826, Appendix F.3).
  • E_vel compares speed at the same frame index. It measures the difference in relative end-effector motion over Δ = 2 frames between generated and reference trajectories (Appendix F.4, Table 17). Both WorldSync variants stay among the top five under E_vel, and expanded coverage lowers E_vel for all six baselines, as it does for raw NDTW.
  • The two metrics are complementary. In a controlled perturbation test (Table 18), E_vel detects a pause followed by catch-up, which NDTW does not. E_vel does not see a constant 2 cm offset, and under an accumulated drift of up to 5 cm it stays below the value of the best model while NDTW rises to that of a mid-ranked one. Where the end effector ends up determines contact and grasp outcomes, so NDTW remains the primary trajectory metric.

Question: Selection of the gate thresholds and penalty, and ranking stability

Short answer. The thresholds were set leniently, and κ was set so that a visually failed rollout never scores better than a valid one. The same three configurations lead in every nearby setting; their order is close and depends on κ.

  • τ_q = 0.33 and τ_m = 0.45. The visual gate mainly screens out rollouts for which action following cannot be meaningfully assessed. The thresholds are not intended as a sharp quality boundary, above which a rollout is necessarily good and below which it is necessarily bad. We therefore adopt a lenient policy and prefer retaining a borderline rollout to rejecting a valid one. The comparison with human labels (Table 16) is consistent with this choice: the four checks retain 90–100% of human-passed clips and reject 50–80% of human-failed clips.
  • κ = 0.3. The penalty follows one rule: a visually failed rollout should not score better than a valid one, so κ is set above the raw NDTW of the evaluated rollouts. The same thresholds and penalty are applied to all models.

Rebuttal Table 3. Gated error (rank among the 14 configurations) of the three leading configurations, varying one parameter at a time; full results in Tables 14 and 15.

SettingWorldSync w/ pretrainWorldSyncDreamDojo (Expanded)Kendall τ vs. original
Original0.0228 (1)0.0234 (2)0.0237 (3)1.000
τ_q = 0.300.0228 (1)0.0232 (2)0.0237 (3)0.978
τ_q = 0.360.0233 (1)0.0238 (3)0.0237 (2)0.978
τ_m = 0.400.0227 (1)0.0230 (2)0.0237 (3)0.978
τ_m = 0.500.0238 (1)0.0252 (3)0.0244 (2)0.978
No q/m checks0.0227 (1)0.0229 (2)0.0235 (3)0.934
κ = 0.10.0193 (3)0.0193 (2)0.0176 (1)0.736
κ = 0.20.0211 (2)0.0213 (3)0.0207 (1)0.890
κ = 0.40.0246 (1)0.0254 (2)0.0267 (3)0.956
κ = 0.50.0264 (1)0.0275 (2)0.0297 (3)0.934
  • The ranking is stable under nearby thresholds. Kendall τ against the original ranking is 0.978 for each threshold change, and WorldSync with pretraining ranks first in all of them. WorldSync without pretraining and expanded-coverage DreamDojo exchange ranks 2 and 3 in two settings, where they differ by less than 0.001.
  • For κ ≥ 0.3, the range consistent with the rule above, the order of the top three does not change. This also holds at κ = 0.75 and 1.0.
  • For κ = 0.1 and 0.2, the weight moves from visual pass rate to raw NDTW. Expanded-coverage DreamDojo, with lower raw NDTW and a lower pass rate than WorldSync, then ranks first. The crossover points are κ ≈ 0.23 and κ ≈ 0.27 against WorldSync with and without pretraining.
  • The main conclusions do not depend on these choices. Expanded coverage lowers the gated error of all six baselines in every setting.

Revision. Sections 3.3.2–3.3.4 and the new Appendices F.2.3 and F.4 contain these additions. Section 4.2 describes the three leading configurations as close in gated error.

We hope these additions address your concerns about soundness.

AuthorsOpenReview ↗

A Brief Summary of Our Responses

Dear Reviewer p5K5,

Thank you again for your careful and constructive review, which has further strengthened our paper. As the discussion period draws to a close, we would be grateful to hear whether our responses address your concerns or whether further clarification would be helpful. We would be happy to respond promptly. If you find that the new evidence addresses your concerns, we would greatly appreciate your consideration of an updated rating, which would be the greatest recognition of our work.

Following your suggestions, we have conducted additional experiments and analyses on component contributions, the link between WorldEcho scores and policy gains, and the robustness of our metrics. All changes are marked in blue in the revised paper. For your convenience, we summarize the key findings below.

  • Weakness 1 (downstream evidence). We added the expanded-data CtrlWorld baseline and a component ablation across seven world models. The analysis shows that data coverage and the two objectives both contribute to policy learning. WorldSync clearly outperforms the stronger baseline, IE and AFE each bring consistent gains beyond expanded data, and lower gated error goes with larger policy gains (Section 4.3).
  • Weakness 2 (frame-aligned signal). We added a frame-aligned velocity error and analyzed its relation to NDTW through controlled perturbations. The two metrics rank the models consistently, while NDTW captures where the end effector goes and the velocity error captures timing, so they are complementary (Appendix F.4).
  • Question (thresholds and penalty). We analyzed the ranking under nearby thresholds and penalties. The ranking is stable, the same leading configurations hold in every nearby setting, and the penalty follows a simple rule that a visually failed rollout never scores better than a valid one (Appendix F.2.3).

We are sincerely grateful for your insightful suggestions, which have been invaluable to our work, and we look forward to hearing from you.

The Authors

Reviewer_QnvCOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper tries to improve action following in action conditioned video world models. It proposes a pipeline of diagnostics (WorldEcho), improvement (WorldSync), and validation with policy improvement using the world model. To evaluate action following, they incorporate demonstrated and off-expert action queries replayed in simulation to obtain ground-truth futures, scored by visual integrity and SE(3) end-effector trajectory alignment. In WorldSync, they broaden action coverage with simulated expert and off-expert trajectories, ground video representations in robot dynamics with an Action-Forcing Expert, and align predicted and ground-truth changes under action interventions. They finally tested the improved policies in simulation and four real-robot tasks.

Critical strengths
  • The paper tackles a critical problem. Action conditioned world models are used as simulators for policy evaluation and improvement, yet whether they follow off-expert actions has not been systematically tested against ground truth, and this is a main bottleneck for policy improvement.
  • The design is novel and the experiments are thorough. WorldEcho combines a visual-integrity gate with SE(3) trajectory alignment against simulator-replayed references, and WorldSync adds feature-level (AFE) and intervention-level (IE) supervision. The evaluation spans six world models, component ablations, and policy improvement in simulation and on four real-robot tasks across three platforms.
Critical weaknesses
  • W1: The policy-improvement experiments lack a key ablation. WorldSync is compared only with CtrlWorld trained on expert data, so it is unclear whether AFE and IE improve policy learning beyond the expanded training data, which alone lowers CtrlWorld’s gated error from 0.0379 to 0.0293. With extra ablation studies, I'm willing to raise to clear accept.
Questions for authors
  • Q1: Can you please address W1?
3 comments
AuthorsOpenReview ↗

Response to Reviewer QnvC

Thank you for the encouraging assessment and for stating clearly what would strengthen the paper. We have run the ablation you asked for.

Weakness 1 / Question 1: Do AFE and IE improve policy learning beyond the expanded training data?

Short answer. Yes. With the training data held fixed, AFE and IE raise policy success after three rounds from 63% (expanded-data Base) to 70% (WorldSync), and WorldSync stays 9 points above CtrlWorld trained on the same expanded data (61%).

We added expanded-data CtrlWorld and the component ablation under the protocol of Section 4.3, extended to three rounds. The full results are in Rebuttal Table 1 of the General Response (2/2); the rows most relevant to your question are:

GroupWorld modelDataGated error ↓Initial (%)Round 2 (%) ↑Round 3 (%) ↑
BaselineCtrlWorldExpert0.0379525756
CtrlWorldExpanded0.0293525961
AblationBase (no IE, no AFE)Expanded0.0264525863
Base + IEExpanded0.0214526367
Base + AFEExpanded0.0232526266
WorldSync (IE + AFE)Expanded0.0193526570
  • Expanded data alone improves policy learning, as you anticipated. It lifts CtrlWorld from 56% to 61% and our backbone without IE and AFE from 56% to 63%.
  • AFE and IE add a further 7 points on the same data. IE alone adds 4 and AFE alone adds 3. All three variants are above expanded-data Base after every round, and full WorldSync is the best variant after Rounds 2 and 3.

Revision. Section 4.3 (Table 2) and Appendix C.3 (Table 8) now report all seven conditions over three rounds and state the gain from expanded data separately from the gain from IE and AFE.

You kindly noted that you would be willing to raise your rating given the additional ablations. We hope these results address your concern, and we are happy to provide further analysis.

Reviewer_QnvCOpenReview ↗

Concern addressed

Thank you for your comprehensive response and experiments. They addressed my concern. I have raised my rating.

We are glad that our response and experiments addressed your concern, and we sincerely appreciate you raising your rating to 4 (Clear acceptance). Thank you also for acknowledging the importance of the problem, the novelty of our design, and the thoroughness of our experiments. Your suggested ablation has substantially strengthened the paper. Thank you again for your support.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response (1/2)

We thank all three reviewers for their careful reading and constructive feedback. The reviewers recognized the following strengths:

  • Important problem: "a critical problem" and "a main bottleneck for policy improvement" (QnvC); "well-motivated and substantial evaluation" (EiuE).
  • Convincing benchmark: simulator-generated counterfactual data is "fully reasonable" and the Figure 5 results are "convincing" (p5K5); "tests action following against action-specific simulator outcomes, rather than relying solely on visual similarity" (EiuE).
  • Coherent pipeline: "the design is novel and the experiments are thorough" (QnvC); "a coherent connection from diagnosis to intervention and downstream validation" (EiuE).

The reviewers also converged on one main request: a downstream policy-improvement ablation that separates the contributions of AFE and IE from expanded data alone. This is a valuable control, and we have run it. We answer it once in part 2/2 of this response and address the remaining points in the individual responses.

Summary of New Results

  1. Downstream component ablation (all reviewers). With the training data held fixed, IE and AFE each raise policy success, and after three rounds their combination raises it most. From a shared 52%, expanded-data Base reaches 63%, Base + IE 67%, Base + AFE 66%, and full WorldSync 70% (Rebuttal Table 1). WorldEcho gated error and policy success are strongly but not exactly related: the two orders coincide after three rounds and differ in earlier rounds.
  2. Expanded-data CtrlWorld as a policy-improvement baseline (QnvC, p5K5). Expanded data lifts CtrlWorld from 56% to 61% after three rounds. WorldSync remains 9 points above this stronger baseline (Rebuttal Table 1).
  3. Frame-aligned velocity error alongside NDTW (p5K5). A new frame-aligned diagnostic ranks the 14 configurations consistently with raw NDTW (Spearman ρ = 0.92), and a controlled perturbation test shows that the two metrics are complementary (Tables 17 and 18).
  4. Sensitivity to gate thresholds and penalty (p5K5). Under nearby thresholds the ranking is stable (Kendall τ ≥ 0.978) and the same three models lead in every setting; we report how the order within them depends on κ (Rebuttal Table 3; Tables 14 and 15).
  5. Discussion of the MBRL literature and model utility (EiuE). Sections 2.2 and 5 now relate our findings on data coverage, accuracy under the policy's distribution, and rollout length to Ross & Bagnell (2012), Wang et al. (2023), and Janner et al. (2019); see the response to Reviewer EiuE.

Summary of Revisions

The revised paper is uploaded, with all changes marked in blue. Table and figure numbers in our responses refer to the revised paper; "Rebuttal Table" denotes a table shown in these responses.

#RevisionLocationRequested by
1Downstream component ablation over three rounds (Rebuttal Table 1)Table 2 in Sec. 4.3; Tables 8 and 9 in Appendix C.3; cross-reference in Sec. 4.4QnvC, p5K5, EiuE
2Expanded-data CtrlWorld policy-improvement baselineTable 2 in Sec. 4.3; Table 8 in Appendix C.3QnvC, p5K5
3Frame-aligned velocity errorSec. 3.3.3; new Appendix F.4 (Tables 17 and 18)p5K5
4Threshold selection and sensitivity analysis (Rebuttal Table 3)Sec. 3.3.2, Sec. 3.3.4; new Appendix F.2.3 (Tables 14 and 15)p5K5
5Discussion of related MBRL workSec. 2.2, Sec. 5EiuE
6More precise claims: the relation between WorldEcho scores and downstream gains is stated as a strong but not exact association under the evaluated protocol; the gain attributable to data is stated separately from that of IE and AFE; the three leading configurations of Table 1 are described as closeSec. 4.2, Sec. 4.3, Sec. 5p5K5, QnvC

We thank the reviewers again for their time and for suggestions that have made the paper stronger. We are happy to answer any further questions during the discussion period.

AuthorsOpenReview ↗

General Response (2/2)

Downstream component ablation: do IE and AFE improve policy learning beyond expanded data?

Raised by Reviewer QnvC (Weakness 1 and Question 1), Reviewer p5K5 (first weakness), and Reviewer EiuE (weaknesses and question).

Short answer. Yes. With the training data fixed at Expanded Action Coverage, adding IE and AFE raises policy success after three rounds from 63% to 70%. Expanded data alone accounts for a gain of similar size (56% to 63%).

Setup. Same protocol as Section 4.3 (Bin Dumping; Appendix C.2): the same initial policy (52%), budgets, and 100 evaluation seeds, with only the world model varied. All conditions are extended from two rounds to three.

Rebuttal Table 1. Policy success rate (%) on Bin Dumping for the initial policy and after each improvement round under matched budgets, with the WorldEcho scores of each world model. Gain is the change in success rate from the initial policy to Round 3, in percentage points.

GroupWorld modelDataInitialRound 1Round 2Round 3GainGated error ↓Raw NDTW ↓Visual pass (%) ↑
BaselineCtrlWorldExpert52545756+40.03790.026695.83
CtrlWorldExpanded52555961+90.02930.021096.97
AblationExpert BaseExpert52545856+40.03520.031498.57
BaseExpanded52555863+110.02640.020497.86
Base + IEExpanded52606367+150.02140.017698.57
Base + AFEExpanded52576266+140.02320.017297.86
WorldSync (IE + AFE)Expanded52576570+180.01930.013897.86

WorldEcho scores of the Ablation rows are from Table 3 (four-task ablation set) and those of the Baseline rows from Table 1 (full benchmark), so they should be compared within each group, not across the two.

  • Expanded data alone improves downstream policy learning. After three rounds it lifts our backbone from 56% to 63% and CtrlWorld from 56% to 61%. The difference appears in Round 3, where both expert-trained models stop improving while all five expanded-data models continue.
  • IE and AFE each add a further gain on the same data, and their combination adds the most. After three rounds IE adds 4 points over expanded-data Base, AFE adds 3, and the two together add 7 (63% to 70%). All three variants are above Base after every round, and WorldSync stays 9 points above expanded-data CtrlWorld.
  • WorldEcho gated error is strongly, but not exactly, related to downstream gain. Within each group, the order by gated error equals the order by success rate after Round 3. Earlier rounds have exceptions: after Round 2, Expert Base and Base are tied at 58%; after Round 1, Base + IE (60%) is ahead of WorldSync (57%) despite its higher gated error. Across the five models of the ablation group, the Spearman correlation is −0.82, −0.97, and −1.00 after Rounds 1, 2, and 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.