acceptodds
Under review as a conference paper at ICLR 2027

ReWorld: Adaptive Corrective Simulation from World Models for Embodied Policy Learning

Abstract

Robotic policies trained from successful demonstrations receive little supervision for recovering from off-nominal states, allowing execution errors to compound into task failure. Physical corrective collection is costly, and task-specific simulators may be unavailable. We introduce ReWorld, which uses an action-conditioned world model as a local simulator to synthesize adaptive corrective experience around demonstrations. ReWorld probes the base policy at predicted perturbation endpoints and retains regions from which it repeatedly fails to recover. For retained cases, a motion planner returns the robot toward the source demonstration and the expert suffix completes the corrective action chunk. The predicted observation and constructed action form corrective training pairs, while competence-aware allocation concentrates supervision on weaker tasks. The world model and planner are used only offline, leaving deployment unchanged. Across 50 RoboTwin 2.0 tasks, ReWorld improves SmolVLA from to and from to . Controlled studies show that selected corrections are more data-efficient than unfiltered corrections, local recovery probes track execution difficulty, and recovery transfers to held-out perturbation magnitudes. Code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, world models, vision-language-action models, corrective learning, embodied AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TAhDOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

ReWorld uses a world model to generate observations of local execution errors and pairs them with corrective trajectories constructed through motion planning and demonstration reuse. These synthetic recovery examples are then used to fine-tune robot policies to improve recovery from execution errors.

Critical strengths

The motivation and central idea are clear, and the overall method is technically reasonable. The experiments support the effectiveness of the proposed approach across different policy backbones on real-world tasks.

Critical weaknesses
  1. Clarity of the technical presentation. The method is difficult to follow, with several terms and notations introduced without sufficient explanation of their purpose. For example, although Section 3.4 and Appendix B.3 describe the sampling probabilities and weighting coefficients for competence-aware post-training, they do not clearly assemble these components into an explicit training objective or step-by-step procedure.

  2. Limited innovation of competence-aware post-training. This component appears to primarily involve task-dependent sampling and loss weighting based on the base policy’s competence. While the experiments demonstrate its utility, its methodological contribution appears incremental.

  3. Insufficient justification of perturbation sampling. The paper does not adequately explain how perturbations are sampled or how their distribution reflects errors encountered during real-world policy execution. Identifying perturbations that the policy cannot recover from does not, by itself, establish their relevance to realistic rollout failures. Further justification of the sampling strategy, along with visualizations and evaluation of the synthetic observations and their consistency with the recovery labels, would strengthen the paper.

Minor Issues

  • CuRobo should be cited at its first mention in the main text, around line 252.
  • The budget parameters in Eq. (3), around line 213, should be defined when first introduced.
Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Response on Core Novelty and the Offline Learning Loop

Response to Reviewer TAhD, Comment 1 of 3

A. Central methodological novelty (W2)

Thank you for recognizing the clear motivation, coherent design, and consistent improvements of ReWorld. We appreciate your close examination of novelty because it highlights the distinction between the full learning framework and an auxiliary weighting rule. ReWorld's central contribution is a usable, closed-loop world-model and policy training framework in offline data space, where counterfactual generation, feedback from the policy's recovery behavior, and corrective supervision jointly guide policy improvement. Competence-aware weighting supports this loop but is not its defining novelty.

The challenge is to use a world model for policy-learning feedback, not just plausible images. From a successful demonstration, the framework must expose recovery failures, construct meaningful corrective targets, and learn from them. ReWorld connects these steps through three operations.

1. Action grounded counterfactual generation. Starting from successful demonstrations, the action-conditioned world model predicts the local visual consequences of structured execution deviations. Unlike arbitrary image augmentation, this associates the altered observation with an explicit perturbed action sequence and the corresponding robot state. Because generation is conditioned on the target scene appearance, it also avoids requiring a fully specified scene-specific simulator for every new layout.

2. Policy conditioned recovery diagnosis. A plausible perturbation is not necessarily a useful corrective training example. Some deviations can already be handled by the current policy, while others reveal an actual weakness. ReWorld measures local recovery outcomes using the base policy and selects perturbation units that expose these weaknesses. This makes data generation adaptive to the policy being improved, rather than depending only on a fixed perturbation distribution.

3. Automatic corrective action labeling. Selected counterfactual observations are paired with recovery actions. The planner returns the robot toward a reference point in the successful demonstration, and the corresponding expert continuation completes the corrective action label. Feasibility and validity checks select examples suitable for offline training. This final step is essential because identifying an error alone does not provide the action supervision needed to correct it.

This is how ReWorld forms a world-model and policy feedback loop for offline post-training. The frozen world model generates action-conditioned deviations, the policy identifies which deviations remain difficult, and planning plus demonstration reuse produce corrective labels that improve the policy. The updated policy can be probed again in a subsequent round. Unlike an online world-model controller, the loop operates during data construction and training, and deployment still uses the original policy interface. CUPID primarily addresses training-data selection, while Ctrl-World studies imagined experience. ReWorld combines local imagination with policy-specific recovery diagnosis and executable corrective supervision.

The controlled experiments offer evidence that this integration matters. In Figure 3(a), 1,000 policy selected corrective samples achieve 80.12% task success, approaching the 80.44% obtained using 4,000 unfiltered corrective samples. The result indicates that recovery aware selection can make corrective supervision substantially more data efficient. The competence allocation study in Table 2(b) addresses a different question, how to distribute that selected supervision across tasks. Under the compared corrective budget, adaptive allocation maintains 68% success for the low competence group while improving the high competence group from 82% to 91%.

Across 50 RoboTwin tasks, ReWorld improves π0.5\pi_{0.5} from 75.64% to 83.32% and SmolVLA from 38.72% to 47.76%. Four real-robot tasks also improve. The deployment interface remains unchanged, with no world model or planner required for inference.

The main takeaway is the complete offline world-model and policy learning loop, not a standalone sampling heuristic. Action-grounded counterfactuals reveal local recovery gaps, policy feedback prioritizes them, and executable labels turn them into corrective learning signals. Competence-aware allocation then balances this supervision across tasks. This is distinct from simply adding synthetic observations or reweighting existing demonstrations.

AuthorsOpenReview ↗

Response on Competence-Aware Training and Implementation

Response to Reviewer TAhD, Comment 2 of 3

B. Training objective and implementation (W1)

The sampling distribution and loss coefficient form one coherent training objective and control two complementary aspects of corrective learning. The sampling distribution determines which tasks receive corrective examples, while the global loss coefficient determines how strongly corrective examples contribute relative to nominal demonstrations.

For task ii, let pip_i denote the estimated competence of the base policy, calculated from task successes on an independent calibration set using the Laplace smoothed estimate pi=(ni++1)/(ni+2)p_i=(n_i^++1)/(n_i+2). We define its desired corrective supervision share as

ci=clip⁡(cmin⁡+α(1−pi),cmin⁡,cmax⁡).c_i=\operatorname{clip}\bigl(c_{\min}+\alpha(1-p_i),c_{\min},c_{\max}\bigr).

Lower competence produces a higher corrective target within the clipping range. We then define ωi=ci/(1−ci)\omega_i=c_i/(1-c_i) and use different task distributions for nominal and corrective examples,

P_C(i)=\frac{\omega_i}{\sum_{j=1}^K\omega_j}.$$ Nominal supervision remains uniform across tasks, while corrective examples concentrate on weaker tasks. The resulting post-training objective is $$\mathcal L(\theta)=\mathbb E\left[ \sum_{n=1}^{b_N}\ell_\theta(z_n^N) +\lambda_C\sum_{m=1}^{b_C}\ell_\theta(z_m^C) \right],$$ where $b_N$ and $b_C$ denote the nominal and corrective sample counts, and $\ell_\theta$ is the policy backbone's native action learning loss. The corrective coefficient is $$\lambda_C=\frac{b_N}{Kb_C}\sum_{j=1}^K\omega_j.$$ Under the stated task sampling distributions, this coefficient makes the expected **loss coefficient share**, rather than the measured gradient share, of corrective supervision for task $i$ equal to its target $c_i$. The per-example losses are combined with a common normalization across the batch. This preserves the relative weighting used in the derivation, while avoiding separate group-wise averaging that would change the effective nominal and corrective balance. Operationally, each training update samples nominal task identities from $P_N$, corrective task identities from $P_C$, draws examples from the corresponding task caches, and computes their native per-example action losses. The corrective losses are multiplied by $\lambda_C$ before the common batch reduction. For the 50 task $\pi_{0.5}$ experiment, the nominal-to-corrective sample ratio is **3 to 1**, with $K=50$ and $\lambda_C=0.4856$. The competence dependent targets range from **0.0641 to 0.3312**, as determined by the observed task competence scores. These settings implement the derivation directly, and the complete objective and training procedure are now consolidated in Appendix B.3. Thus, competence-aware allocation preserves nominal supervision while targeting weaker tasks. It complements, rather than replaces, the core corrective-data construction mechanism. ## C. Minor presentation details We have added the **CuRobo citation at its first occurrence in Section 3.3**. We have also clarified beside Eq. (3) that **$B_{\min}=48$** and **$\Delta B=96$** are the base and competence-dependent additional probing budgets. Appendix B.3 now presents the complete loss and the corresponding task sampling implementation together.
AuthorsOpenReview ↗

Response on Perturbation Design and Natural Failure Coverage

Response to Reviewer TAhD, Comment 3 of 3

D. Perturbation design and natural-failure coverage (W3)

We appreciate the request to clarify why the perturbations are meaningful for embodied execution. The perturbations represent three common classes of local control deviation, positional misalignment during reaching or placing, rotational misalignment during end-effector orientation control, and premature gripper closure during manipulation. These are action-level deviations with direct consequences for what the robot observes and which corrective action it needs next. They are not arbitrary appearance changes.

To make the perturbation design concrete, translation errors are sampled along the six Cartesian directions, with magnitudes of 3.3, 6.7, and 10 cm. Rotation errors use magnitudes of 5, 10, and 15 degrees. Premature gripper closure represents an action timing error that can interrupt a grasp before the policy reaches its intended manipulation state. Perturbations are introduced at demonstration-aligned moments within approach, pregrasp, transport, and place phases. Their purpose is to create a controlled, reproducible family of local deviations whose physical meaning is clear.

Importantly, sampling a perturbation is not the same as retaining it for training. ReWorld subsequently performs policy-conditioned recovery diagnosis. Each perturbation unit is evaluated with four base policy recovery probes. Position errors are evaluated against a 3 cm local alignment criterion, rotation errors against a 10 degree orientation criterion, and premature gripper closure against successful gripper reopening. A future expert state window allows the probe to recognize realignment with the demonstration without requiring an exact return to one timestamp. A perturbation unit is retained only when the empirical recovery rate is at most 25%, meaning the base policy succeeds in at most one of its four probes.

The perturbation families specify interpretable local errors, while the policy probes determine which instances reveal recovery weaknesses. Retained units proceed to corrective labeling only if feasible recovery and expert continuation are available. The revised supplement reports full 50-task retention statistics by type, magnitude, and phase.

Natural failure evidence and corrective label validity

To assess whether these perturbation families reflect errors encountered during ordinary policy execution, we examined 400 naturally failed base policy rollouts. This study complements controlled perturbation probing by asking which early local deviations actually arise before a natural task failure, and how often those deviations correspond to ReWorld's three structured error families.

Observation from natural failuresCountRate
Identifiable early local execution deviation307 / 40076.8%
Early deviation covered by translation, rotation, or premature gripper closure249 / 40062.3%
Covered among rollouts with identifiable early local deviations249 / 30781.1%

The key result is that 81.1% of identifiable early deviations belong to these three physical error families. Natural failure coverage establishes the relevance of the perturbation design. Policy-conditioned probing then identifies the particular deviations that remain difficult for the current policy.

Finally, the generated observations and corrective labels were independently examined on 500 accepted samples from 10 RoboTwin tasks. Target object mask consistency reached 94.8%, and 472/500 corrective trajectories, or 94.4%, passed the simulator complete task success criterion. These results complement the selection statistics by assessing whether the perturbed observations and associated recovery actions can be used as executable training supervision. Full selection breakdowns and validation details are included in the revised supplementary material.

Together, the natural-failure analysis, policy-conditioned selection, and complete-trajectory validation connect the error families encountered in execution to automatically constructed, executable corrective supervision.

Reviewer_ieY2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ReWorld, a method that uses a world model to generate corrective training data for robot policies. Starting from successful demonstrations, it introduces errors such as position shifts, rotations or premature gripper closure, then predicts the resulting observations. The base policy is tested on these states to identify cases it struggles to recover from. Corrective actions are then constructed using motion planning and the remaining demonstration actions. These examples are mixed with the original demonstrations for further training, with more corrective supervision assigned to weaker tasks. The world model and planner are used only offline, so deployment uses the trained policy directly. On 50 RoboTwin 2.0 tasks, ReWorld improves success rates from 38.72% to 47.76% for SmolVLA and from 75.64% to 83.32% for pi_05. Across four real-robot tasks, average success increases from 59.5% to 68.5%.

Critical strengths
  1. The paper addresses a meaningful problem with a clear motivation. Successful demonstrations provide limited supervision for recovering from execution errors, while collecting corrective trajectories on physical robots is costly. Using a world model to generate local deviations around successful demonstrations offers a practical way to address this gap.

  2. The corrective-data selection is well motivated. ReWorld probes the base policy to identify recovery weaknesses and directs data generation toward those cases. Comparisons at matched sample counts show that selected corrections are more useful than unfiltered corrections. The relationship between local probe results and full-task difficulty, together with the competence-aware allocation ablations, provides further support for the design.

  3. The approach is practical and empirically supported. The world model and motion planner are used only during offline data construction, preserving the original policy’s deployment interface. Experiments across two policy backbones, 50 simulated tasks and four real-robot tasks demonstrate the method’s utility. Additional tests with unseen perturbation magnitudes and another world model provide evidence that the benefits extend beyond a single configuration.

Critical weaknesses
  1. It is unclear how accurately the world model predicts the effects of perturbations. The authors acknowledge that generated observations may contain errors, but do not measure how often these occur. For example, does the robot shown in the generated image match its calculated pose? Do objects move and make contact correctly? How many generated samples are rejected? Better task success shows that the synthetic data is useful, but does not tell us how reliable these predictions are.

  2. Returning the robot to its original pose may not make the original demonstration usable again. A perturbation may move an object, cause it to fall or change how it is held. Even if the robot returns to the demonstration pose, the next demonstrated action may no longer work. The paper includes checks before accepting corrective samples, but does not clearly explain how these checks detect such changes.

  3. Passing the local recovery check does not always mean the task can continue successfully. Figure 3(b) shows that the checks are informative, but their mistakes are not examined in detail. For example, a perturbation could push a cup away. The robot might then return within 3 cm of the demonstrated position and pass the check, yet still grasp empty space because the cup has moved. It remains unclear how often such cases occur and whether they cause useful corrective examples to be missed.

  4. Some key steps are difficult to follow. Although the overall pipeline is understandable, the paper could more clearly distinguish the reference states used for recovery probing and corrective-action construction. It is also difficult to follow how the generated observation, calculated robot state and corrective action are combined into a training sample. A concrete example showing one demonstration through the full pipeline would improve readability.

Questions for authors
  1. Could the authors compare generated observations with what actually happens in the simulator under the same perturbations? This could show whether robot poses, object movements and contacts are predicted correctly, and how often generation fails or samples are rejected.

  2. How does the method check whether the original demonstration can still be followed after an object moves, falls or slips from the gripper? Does the motion planner use updated object positions?

  3. Could the authors show examples where the local recovery check passes but the full task fails, and report how often this happens?

  4. How are perturbations applied within an action chunk, and how is the resulting robot state computed?

  5. How are the recovery motion and expert continuation joined, and what truncation or padding rules are used to match the horizon H?

3 comments
AuthorsOpenReview ↗

Response on World-Model Fidelity and Corrective Validity

Response to Reviewer ieY2, Comment 1 of 3

Thank you for recognizing ReWorld's practical motivation, the policy-conditioned selection mechanism, and the gains across simulation and physical tasks. Your questions about world-model fidelity and object-state changes address a central requirement of our contribution, namely that a synthetic observation must be paired with a corrective action that is meaningful from the perturbed state. We therefore report visual consistency and complete corrective-trajectory validity as separate measurements, rather than treating one as a proxy for the other.

A. World-model observation accuracy (W1, Q1)

ReWorld uses the world model for short-horizon, action-conditioned counterfactual prediction around successful demonstrations. From a source observation, robot state, and perturbed action chunk, it predicts the visual endpoint that the policy would encounter after a local deviation. The robot state is computed from that same perturbation. This is a more focused use of the world model than generating an entire imagined manipulation episode.

We compared 500 accepted samples from 10 RoboTwin tasks using π0.5\pi_{0.5} with simulator references under matched scenes, camera views, and perturbations.

Visual measureResultWhat it tests
Mean end-effector reprojection error4.2 pxRobot-image alignment with the perturbed robot state
Target-object mask consistency, IoU ≥0.75\geq 0.7594.8%Agreement of generated and simulator-reference object regions

The reprojection result indicates close alignment between the robot represented in the image and the state computed from its actions. Object-mask consistency shows that relevant scene appearance is maintained for most generated endpoints under the specified criterion. These measurements support the local visual fidelity of the synthetic input, while contact-state compatibility must still be evaluated through task execution.

B. Object-state changes and expert-suffix validity (W2, Q2)

A recovered robot pose does not automatically restore a displaced object. ReWorld therefore focuses on early local correction, where a feasible recovery prefix can rejoin the source demonstration and the expert continuation can remain useful. Candidate policy re-probing, motion planning, collision and joint-limit checks, and suffix validation are applied during data construction. CuRobo can incorporate updated object poses into collision checking. For the direct validity test, we replayed each perturbation in RoboTwin and executed the entire recovery prefix and expert suffix.

Simulator-grounded validationResult
Task context preserved485/500, 97.0%
Significant object or contact change15/500, 3.0%
Complete corrective trajectory valid472/500, 94.4%
Validity with preserved context465/485, 95.9%
Validity with significantly changed context7/15, 46.7%

The high 94.4% full-trajectory validity is the central result because it measures the actual usefulness of the corrective label, rather than visual similarity or a successful robot-pose return alone. Within the context-preserved group, validity reaches 95.9%. The lower 46.7% rate for altered contexts demonstrates why the task state matters, while the altered group represents only 3.0% of the accepted validation samples.

In a separate matched-budget consistency ablation, a consistency-filtered variant reaches 76.6% task success, compared with 76.4% for standard ReWorld under the same evaluation setup. The 0.2 percentage point difference suggests that excluding these rare altered-context samples has little measurable impact in this comparison. Separately, the 10-task selection analysis finds significant changes in 26/861 accepted samples, 3.0%, versus 27/138 rejected candidates, 19.6%. Thus, the generation and filtering pipeline preferentially retains compatible task contexts, and full simulator replay confirms that most constructed corrective trajectories can complete the task.

AuthorsOpenReview ↗

Response on Local Recovery Probes and Task Success

Response to Reviewer ieY2, Comment 2 of 3

C. Relationship between local recovery probes and complete task success (W3, Q3)

The local recovery probe and the full-task evaluation answer different questions. The probe asks whether the base policy can realign with a nearby successful demonstration after a specified perturbation. This produces a practical signal for selecting corrective training examples. It is not used as a guarantee that the entire subsequent task will succeed.

The paper already provides evidence that the local signal is informative about broader behavior. In Figure 3(b), full-task success increases from 22% to 78% across bins with increasing local probe pass rates. Furthermore, ReWorld yields gains of 25 and 23 percentage points in the two hardest recovery groups. These findings suggest that training specifically on weak local recovery states is useful for improving subsequent task execution.

To examine imperfect correspondence between local and global outcomes directly, we performed an additional analysis on 400 independently perturbed states. Each state was evaluated for the local recovery criterion and for subsequent complete task success.

Outcome conditioned on local probe resultFull task result
Full task success given a probe pass67.6%
Full task success given a probe failure43.5%
Full task failure despite a probe pass32.4% of probe passes

The first two rows show that a successful local recovery probe is associated with a substantially higher probability of completing the task. At the same time, the third row quantifies precisely the cases highlighted in the review, where local realignment is achieved but later task execution still fails. The 32.4% value is conditional on probe passes. It should not be interpreted as a failure rate over all 400 perturbed states.

This distinction is useful for understanding ReWorld's objective. The method does not train the policy to predict global success from one local check. Instead, the check identifies states in which corrective supervision is likely to be valuable because the current policy has difficulty returning to successful behavior. To see whether that supervision changes what the policy subsequently does, we compared Base and ReWorld from matched initial deviations.

Closed loop behavior from matched local deviationsBaseReWorld
Local realignment success53.0%63.0%
Subsequent severe execution errors26.5%21.0%

ReWorld improves local realignment by 10.0 percentage points and reduces severe downstream execution errors by 5.5 percentage points. This is consistent with the central learning objective. Better early correction does not eliminate every possible later failure, but it helps prevent local deviations from developing into more damaging execution errors.

AuthorsOpenReview ↗

Response on Corrective Action Construction and Alignment

Response to Reviewer ieY2, Comment 3 of 3

D. Detailed construction of the corrective action chunk (W4, Q4, Q5)

We agree that distinguishing the reference states used for diagnosis and corrective action construction is important for understanding the method. The two stages have related but different purposes, and we clarify them step by step.

1. Applying a perturbation and constructing the associated state. Starting from a nominal demonstration action chunk, ReWorld perturbs the active arm's translational motion, rotational motion, or gripper closing command. In the implementation described in Appendix B.2, translation perturbations use the six Cartesian directions, with magnitudes of 3.3, 6.7, and 10 cm, and rotation magnitudes of 5, 10, and 15 degrees. Applying the perturbation to the nominal action chunk produces AtδA_t^\delta. The corresponding robot state is computed from the same modified sequence, sδ=State⁡(stN,Atδ)s^\delta=\operatorname{State}(s_t^N,A_t^\delta), using the robot state representation and kinematics. The action-conditioned world model predicts the corresponding visual observation o^δ\hat o^\delta from the source observation, nominal robot state, and perturbed actions. This gives the observation and state components of the corrective training tuple from a common intervention.

2. Reference for recoverability diagnosis. In Section 3.2, the base policy is queried from the perturbed observation and state. Its predicted response is compared against a future expert state window, beginning after the perturbed action horizon and extending for the specified local matching interval. This allows the probe to recognize local realignment without requiring the policy to arrive at one exact demonstration timestamp. The result is a binary recovery outcome for that probe, and four such probes determine the empirical recovery rate of a perturbation unit.

3. Reference for constructing the corrective label. In Section 3.3, the planner instead uses the source demonstration configuration at the start of the perturbed segment as the recovery target. This reference supports constructing an executable action prefix that returns the robot toward the demonstration trajectory. For premature gripper closure, the prefix reopens the active gripper while holding the arm pose fixed. The expert continuation is then taken from the corresponding source demonstration segment. Thus, the future matching window used for diagnosis should not be confused with the source reference used for corrective label construction.

4. Aligning recovery and continuation to the action horizon. The corrective label follows Eq. (4),

AC=TH(Pδ⊕AtN,suffix).A^C=\mathcal T_H\bigl(P^\delta\oplus A_t^{N,\mathrm{suffix}}\bigr).

For π0.5\pi_{0.5}, the prediction horizon is H=32H=32 actions. The default construction allocates 16 recovery actions and 16 expert continuation actions to translation and rotation perturbations. Premature gripper closure uses 10 reopening actions and 22 continuation actions. The alignment operator first resamples an overly long recovery prefix to the full horizon so that its final recovery target is preserved. Otherwise, it concatenates the prefix and expert continuation in temporal order, truncates a sequence longer than HH to its first HH actions, and right pads a shorter sequence by repeating its last action. The physical action channels and any padding needed for the policy's action representation are handled separately from this temporal alignment.

The complete tuple (o^δ,sδ,ℓ,AC)\bigl(\hat o^\delta,s^\delta,\ell,A^C\bigr) is then used for offline corrective post-training. In the revised Appendix B.2, these steps are specified together with the trajectory checks and action horizon rules. The resulting training policy retains its original inference interface and does not call either the world model or the planner at deployment.

In summary, your questions identify the precise measurements needed to assess the reliability of world model based corrective supervision. The added observation consistency study, simulator execution validation, local probe analysis, and action chunk specification provide those measurements at distinct levels. They show why the local recovery signal is useful, how it differs from guaranteed task success, and how ReWorld turns difficult but feasible deviations into executable corrective training examples. We hope these clarifications make the method's empirical support and practical scope much easier to evaluate.

Reviewer_sakKOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReWorld, an offline corrective-data generation framework for embodied policy learning. It uses an action-conditioned world model as a local simulator to probe a base policy around successful demonstrations, selects states where the policy fails to recover, and combines a motion-planner return with an expert suffix to construct corrective training pairs. A competence-aware allocation strategy concentrates data on weaker tasks, while the world model and planner are removed at deployment. On 50 RoboTwin 2.0 tasks, ReWorld improves SmolVLA from 38.72% to 47.76% and π0.5 from 75.64% to 83.32%. The paper also reports matched-budget selection studies, perturbation-magnitude transfer, and four real-robot tasks.

Critical strengths
  1. The paper identifies a real structural gap in behavior-cloning data: successful demonstrations specify nominal actions but provide little supervision for locally recovering from execution deviations. The proposed decomposition—use a world model to create a local counterfactual observation, diagnose whether the current policy can recover, and construct a recovery prefix followed by the expert suffix—is conceptually coherent and keeps the deployment interface unchanged. This is a more focused use of a world model than attempting long-horizon imagined rollouts.

  2. The paper makes a useful attempt to separate three sources of improvement that are often conflated in corrective-data work: generating off-nominal states, selecting states that expose policy-specific recovery gaps, and allocating supervision across tasks. The matched-budget selection study, the competence-allocation study, the Base + 10k control, the held-out magnitude evaluation, and the EVAC cross-world-model result provide meaningful evidence that the gains are not explained solely by additional optimization or by simply adding arbitrary synthetic samples.

Critical weaknesses
  1. The central training tuple is not shown to be causally consistent. The predicted observation comes from the action-conditioned world model, the robot state is computed from the perturbed command, and the corrective label is produced by planning back to a source demonstration state and appending the original expert suffix. This construction assumes that returning the robot to the source state also restores the task context. That assumption can fail when a perturbation changes object pose, contact state, grasp status, or other environment variables. The validity checks described in the paper appear primarily kinematic and trajectory-level; without measuring object/contact-state consistency or the rate and consequences of false synthetic corrections, it is unclear whether the method is learning recovery or learning from internally inconsistent observation-action pairs.

  2. The paper’s policy-conditioned diagnosis is a one-time diagnosis of the frozen base policy, rather than an iterative correction procedure. The offline-equivalence argument only establishes equivalence to online generation when the world model, planner, probing policy, and selection table remain fixed; it does not establish coverage of failure states induced by the updated policy after post-training. Moreover, planner and suffix filters retain only states that are both difficult for the base policy and recoverable by the chosen planner, potentially excluding the hardest contact, object-state, and long-horizon failures. Thus the evidence supports local, planner-recoverable re-alignment around demonstrations, but is not yet sufficient for the broader claim of general recovery learning.

Questions for authors
  1. How do the authors establish causal consistency of each retained tuple (o^δ,sδ,ℓ,AC)(\hat{o}^{\delta},s^{\delta},\ell,A^C)? In particular, can they quantify cases in which the world model predicts an altered object/contact state but the planner returns only the robot to the source state, and can they report performance separately on samples with and without object-state changes?

  2. The recoverability table is computed from the frozen base policy and then held fixed. After corrective post-training, do new probes with the updated policy reveal new failure regions or materially different recovery rates? A comparison between one-shot ReWorld and one or more re-probing rounds would clarify whether the proposed offline procedure addresses policy-induced distribution shift or only mines the initial policy’s errors.

  3. What is the acceptance distribution after all filters? Please report the rejection rates and task/phase breakdown for (i) the local recovery probe, (ii) motion-planner failure, (iii) collision or joint-limit checks, and (iv) expert-suffix validity. Do rejected candidates disproportionately correspond to contact changes, object-state changes, or long-horizon failures?

  4. What is the actual role of the action-conditioned world model relative to the policy-conditioned selection and planner-based labeling? Can a simpler observation perturbation or renderer-based local simulator, with the same selection and labeling pipeline, achieve comparable gains? This would determine whether the contribution is fundamentally a world-model method or an adaptive corrective-data construction method.

3 comments
AuthorsOpenReview ↗

Response on Corrective Label Consistency and Validity

Response to Reviewer sakK, Comment 1 of 3

Thank you for recognizing the important gap between successful demonstration learning and recovery from execution errors, as well as the coherent design of ReWorld. We especially appreciate your attention to the central idea, namely turning local counterfactual experience into policy-specific corrective supervision. ReWorld combines action-conditioned observation synthesis, diagnosis of the current policy's recovery weaknesses, and automatic corrective labels. The resulting policy learns early correction offline and does not require an additional recovery system at deployment. We begin with your most fundamental concern, whether this supervision is causally consistent and executable.

A. Causal consistency of corrective supervision (W1, Q1)

The issue is whether a world-model-predicted observation, the robot state implied by a perturbed action, and a planned corrective label describe a compatible physical situation. Returning the robot to a demonstration pose does not itself guarantee that an object or contact state has remained unchanged. We therefore designed ReWorld for early local deviations around successful demonstrations, where the recovery prefix can reconnect with the expert continuation before errors cascade into more substantial scene changes.

To test this property, we evaluated 500 accepted corrective samples from 10 RoboTwin tasks. For each sample, we compared the world-model-generated observation against a simulator reference with the same initial scene and action perturbation. We also replayed the perturbation in the simulator and executed the full planned recovery prefix followed by the expert continuation. This deliberately separates the consistency of the observation from the actual executability of its corrective action label.

End-effector reprojection errorTarget-object mask consistency, IoU ≥0.75\geq 0.75Task context preservedComplete corrective trajectory valid
4.2 px94.8%485/500, 97.0%472/500, 94.4%

The reprojection error measures agreement between robot keypoints in generated images and the projections of the robot state implied by the perturbed actions. The object-mask result measures agreement with simulator-reference object regions. Both assess visual consistency, while the 94.4% full-trajectory validity directly measures whether the corrective label can finish the task when executed from the perturbed physical state. Visual similarity alone is not treated as proof of valid contact or task progress.

We further divided the same validation set by changes in task context. When context was preserved, 465/485 corrective trajectories, or 95.9%, were valid. Among the 15 samples with significant object or contact changes, 7/15, or 46.7%, remained valid. These results show why context preservation matters for expert-suffix execution and also show that significant changes are uncommon among accepted samples.

In a separate matched-budget consistency ablation, the consistency-filtered variant reached 76.6% task success, compared with 76.4% for standard ReWorld under the same evaluation setup. The 0.2 percentage point difference suggests that excluding these rare context changes has little effect on success in this experiment. Together, simulator-reference consistency, complete corrective execution, and the matched-budget ablation provide complementary evidence that the automatically constructed labels supply useful local recovery supervision.

AuthorsOpenReview ↗

Response on Filtering, Coverage, and Iterative Diagnosis

Response to Reviewer sakK, Comment 2 of 3

B. Policy-conditioned filtering and data coverage (W2, Q3)

We first clarify how ReWorld decides which perturbations should become corrective training examples. Recovery probing selects states that the base policy finds difficult, while planner and trajectory checks retain those for which executable corrective labels can be constructed. The two stages serve different purposes and should not be interpreted as one undifferentiated acceptance rate.

In a 10-task analysis, 748 perturbation units were evaluated with four recovery probes each, giving 2,992 trials. A total of 335 units, or 44.8%, met the low-recoverability threshold of at most 0.25. Corrective construction then proceeded through the following candidate-level checks.

StageRemainingRejected at stage
Generated candidates9990
Candidate policy probing96336
Planner feasibility91449
Collision and joint limits88727
Expert-suffix validity86126

Thus 861/999 candidates, or 86.2%, entered the corrective cache after the initial policy-conditioned unit screening. High candidate acceptance is consistent with the earlier elimination of readily recoverable perturbation units. Significant object or contact changes occurred in 26/861 accepted samples, or 3.0%, compared with 27/138 rejected candidates, or 19.6%. This comparison indicates that the complete pipeline preferentially retains stable task contexts. This reflects the combined selection process rather than a rejection rule based solely on object motion.

The revised supplement also reports a broader 50-task unit-level analysis, covering 3,286 units and 13,144 probes, with 1,526 units retained, or 46.4%. Detailed breakdowns by perturbation type, magnitude, manipulation phase, and active arm characterize recovery difficulty across the full benchmark. These 50-task statistics are separate from the 10-task candidate funnel above.

C. One-shot construction and iterative diagnosis (W2, Q2)

ReWorld is not restricted to diagnosing the initial policy once. After one round of post-training, the updated policy can be probed again using the same world model and planner, allowing new corrective data to target residual weaknesses. Our main experiments use one-shot construction for efficiency, while the same procedure supports further rounds of diagnosis and training.

We tested a second round on five RoboTwin tasks.

TaskBaseOne-shotTwo-round
adjust_bottle90%96%98%
grab_roller92%96%96%
handover_mic48%64%68%
press_stapler88%96%96%
stack_blocks_two8%26%32%
Mean65.2%75.6%78.0%

The first round improves mean success by 10.4 percentage points, and the second adds 2.4 points, primarily on tasks that remain difficult. Continued training on the original first-round data produced essentially no further gain, indicating that refreshed diagnosis contributes beyond extra optimization on the unchanged cache.

The second round did not reveal a new category of execution errors. In this five-task study, the remaining failures stayed within the translation, rotation, and premature gripper closure regions already covered by the first round, while the set of difficult cases became smaller after training. This suggests that the remaining improvement potential may lie more in training optimization and policy representation or architecture than in introducing additional perturbation categories. Re-probing can still focus supervision on the smaller residual set, which explains its modest further gain. One round captures most of the observed benefit at lower data-generation cost. The equivalence argument in Appendix A holds for frozen components within a construction round, and permits re-probing an updated policy in subsequent rounds.

AuthorsOpenReview ↗

Response on the Role of Action-Conditioned World Models

Response to Reviewer sakK, Comment 3 of 3

D. Why use an action-conditioned world model? (Q4)

This question goes to the motivation and novelty of ReWorld. The action-conditioned world model is not introduced as a new backbone, and the method does not depend on generating arbitrarily long imagined task trajectories. Its specific role is to provide local, action grounded counterfactual observations around successful behavior, so that policy weaknesses can be diagnosed and paired with corrective actions before deployment.

A simple image transformation can change the appearance of a frame, but it does not by itself describe how the robot and surrounding objects would respond to a particular perturbed action sequence. This distinction matters because the training tuple must link an off-nominal observation, the corresponding robot state, and a corrective action. ReWorld uses an action-conditioned world model to generate the observation associated with a structured action perturbation, while the same perturbation defines the robot state used to build the corrective label.

A task-specific simulator can generate action grounded perturbations, but its sim-to-real gap is especially consequential for corrective manipulation learning. Accurate recovery requires the simulator to capture contact, friction, object properties, and interaction dynamics, which can be difficult to specify for unfamiliar or complex objects. Errors in these physical models, as well as discrepancies in scene geometry and appearance, can make simulated recovery states differ from those encountered by the deployed robot. ReWorld takes a complementary route. Its action-conditioned world model predicts local counterfactual observations from the current scene appearance and perturbed actions, reducing dependence on a manually specified simulator for each object and layout. This approach does not eliminate prediction error or guarantee physically correct contacts, which motivates our local construction and simulator-grounded validity analysis.

Importantly, the contribution lies in what we do with these observations. They are not merely added as synthetic images. We test the current policy on the corresponding perturbations, retain states that reveal its recovery weaknesses, and automatically construct executable corrective action labels. The resulting experience is used for offline post-training, allowing early corrective behavior to be internalized by the policy. At deployment, neither the world model nor the planner is required.

The principal evidence is on the π0.5\pi_{0.5} policy, where ReWorld improves 50-task RoboTwin success from 75.64% to 83.32%, a gain of 7.68 percentage points without changing deployment. The additional improvement on SmolVLA, from 38.72% to 47.76%, supports transfer across policy capacities. As a complementary world-model portability check, Appendix D replaces GE-Sim with EVAC and reports gains for ACT, 55.6% to 67.6%, and SmolVLA, 62.0% to 73.2%, over five tasks. This auxiliary experiment tests an alternative observation-generation backbone, while the main π0.5\pi_{0.5} result directly demonstrates the value for a strong modern policy.

In summary, ReWorld addresses a specific structural limitation of successful demonstration training. It uses world models not as complete replacements for physical interaction, but as local generators of policy relevant recovery experience. Its novelty is the integration of counterfactual observation synthesis, policy-conditioned recovery diagnosis, and automated corrective labeling into an offline learning procedure that improves recovery behavior without changing the deployed policy interface.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Follow-Up on Our Rebuttal and Additional Evidence

Dear Reviewers,

We sincerely appreciate your thoughtful reviews and constructive feedback on ReWorld.

We have provided detailed responses to the concerns raised in the reviews, together with additional experimental evidence and corresponding manuscript revisions. These include validation of corrective supervision, analyses of policy-conditioned data selection, iterative recovery diagnosis, and further clarification of the central methodological contributions.

We believe these additional results strengthen the empirical support for ReWorld and clarify its role as an offline world-model and policy learning framework.

As the discussion period approaches its conclusion, we would greatly appreciate any further feedback or remaining questions you may have. We would be happy to provide additional clarification and continue the discussion.

Thank you again for your time and consideration.

AuthorsOpenReview ↗

A Gentle Request for Reviewer Feedback

Dear Reviewers,

Thank you again for your thoughtful reviews and constructive feedback.

We have carefully addressed the main concerns raised in your reviews and provided additional experimental results, methodological clarifications, and corresponding manuscript revisions.

As the discussion period is approaching its conclusion, we would sincerely appreciate it if you could take a moment to review our rebuttal and consider the additional evidence when revisiting your assessment.

Thank you very much for your time and consideration.

Best regards, The Authors

AuthorsOpenReview ↗

Summary of Our Rebuttal

Dear Reviewers and Area Chair,

Thank you again for your thoughtful reviews and constructive feedback on ReWorld.

As the discussion period comes to an end, we would like to briefly summarize the main concerns we have addressed in our rebuttal.

First, we provided additional validation and analysis to clarify the reliability of world-model-generated observations and the validity of the corresponding corrective supervision, including the consistency between perturbed states, recovery actions, and expert continuations.

Second, we clarified the policy-conditioned selection mechanism, the relevance of our perturbation design to execution failures, and the relationship between local recovery diagnosis and overall task success. We also discussed the potential for iterative diagnosis and further corrective training.

Third, we provided more detailed explanations of corrective action construction and competence-aware training, and clarified the central methodological contribution of ReWorld as an offline feedback loop connecting world-model-generated counterfactual experience, policy-specific recovery diagnosis, and executable corrective supervision.

We believe these responses and the corresponding manuscript revisions have substantially addressed the main concerns raised in the reviews and have helped clarify both the empirical support and the methodological contributions of our work.

We would sincerely appreciate it if you could review our detailed rebuttal and consider these clarifications and additional evidence in your final assessment.

Finally, thank you very much for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.