acceptodds
Under review as a conference paper at ICLR 2027

Correct the Action, Not the Policy: Online Recovery under Dynamics Shifts

Abstract

Changes in object dynamics can cause a pretrained manipulation policy to fail even when object geometry and task goals remain unchanged. We study whether such failures can be recovered during execution without updating the policy itself. We introduce a model-based action correction approach that estimates effective dynamics from interaction records, trains a lightweight predictive model under the estimated dynamics, and modifies only a low-dimensional subset of the frozen policy’s actions. During execution, candidate corrections are evaluated from the current observation and selected to keep predicted motion close to a successful trajectory produced by the same policy under reference dynamics. Across shifts in object mass and contact friction, the method raises success from 0–4% to 52–78% in rope routing and T-pushing, and from 65% to 78% in toy packing. A separate physical rope experiment further shows that a simulation-selected correction can recover an execution that fails after a change in object properties. These results show that some failures caused by dynamics shifts can be recovered through execution-time action correction rather than policy retraining.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, deformable object manipulation, policy adaptation, system identification, residual action correction, model-based planning, distribution shift, differentiable/particle physics simulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AxWWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies recovery of a frozen manipulation policy under dynamics shifts. It identifies effective simulator parameters, trains a dynamics surrogate, and searches for low-dimensional action corrections whose predicted trajectories track a successful reference execution. Experiments on rope routing, T-pushing, and toy packing report improved success over the uncorrected policy, complemented by a physical rope demonstration.

Critical strengths
  1. Relevant and well-defined adaptation setting. The paper addresses failures caused by changed dynamics while retaining the pretrained policy. Restricting adaptation to low-dimensional action corrections provides a concrete alternative to policy retraining.

  2. Substantial recovery in the evaluated settings. Success increases from 0% to 78% under the rope mass shift and from 0% to 74% in T-pushing, with improvements across all four conditions. The physical rope demonstration provides preliminary evidence that a simulation-selected correction can transfer to hardware.

  3. A concrete design for repeated correction. The surrogate enables candidate evaluation without simulator calls during execution. Observation-dependent updates, ensemble disagreement penalties, and correction regularization form a coherent mechanism for adjusting the frozen policy's actions.

  4. Transparent reporting and useful failure analysis. The paper reports stronger baseline results and negative terminal-cost results rather than presenting uniformly favorable comparisons. The gripper diagnostics connect rope-routing failures to premature release, while the discussion distinguishes effective simulator parameters from true physical properties.

Critical weaknesses

1. The practical advantage over the best baselines remains unclear.

The proposed method does not outperform the best baseline in any of the four simulated conditions; the terminal-cost variant only matches the best baseline on T-pushing. Tables A1–A2 report adaptation execution counts, but not total adaptation time or measured online correction latency. Consequently, it remains unclear what efficiency benefit compensates for the success-rate gap.

Please quantify this success–cost trade-off and further analyze baseline failures, particularly residual RL under increased floor friction and direct search in toy packing. Section 6.2 identifies a simulation-to-shifted-physics mismatch for the latter, but does not isolate its cause. These analyses would clarify when the proposed method is preferable.

2. Fixed temporal alignment may conflate delayed progress with incorrect behavior.

Equation 6 compares predicted and reference states at matching time indices. Both dynamics shifts and execution variability can change task progression, so an appropriate but slower execution may be penalized or encouraged toward a premature task phase. No targeted temporal-alignment robustness study is reported. Please evaluate temporal offsets and execution-speed variation, and compare fixed alignment with monotonic progress-based alignment.

3. Discrimination between candidate trajectory outcomes is insufficiently validated.

In toy packing, direct search passes selection under estimated physics but achieves 0% success under shifted physics (Section 6.2), showing that simulated success need not transfer. This does not establish a failure of the proposed surrogate, but motivates testing whether its candidate rankings predict realized outcomes. Section 6.3 validates simulator motion fitting rather than this decision process. Please visualize predicted and realized trajectories for successful and failed candidates, and quantify candidate-ranking agreement or separation of predicted scores by realized outcome.

4. Generalization beyond the reference configuration remains unverified.

The method restores behavior around a previously successful trajectory, with evaluation restricted to fixed scene geometry and task goals. Although consistent with the stated scope, this limits the demonstrated practical applicability. Modest changes in initial object positions or obstacle layouts would help characterize when reference tracking remains useful.

The physical experiment transfers a simulation-selected correction rather than validating observation-dependent online updates. As acknowledged in Section 7, the central online mechanism therefore remains unverified on hardware. Repeated physical trials with online updates would strengthen the practical evidence.

Questions for authors

Mentioned in Weaknesses

3 comments
AuthorsOpenReview ↗

Progress-aware correction, computational cost, and generalization

Thank you for the constructive comments. We have updated the BC policies and re-evaluated all ablations and baselines, with additional analyses of computational cost, temporal alignment, and surrogate accuracy during correction.

1. Recovery performance and computational cost

The submitted BC policies used a time index and were trained for fixed initial configurations. We retrained BC with demonstrations from varied initial object/robot poses, completely removed the time-index input, and retained observation-history inputs. All objective ablations and adaptation baselines were re-evaluated using these updated BC policies and the same evaluation conditions within each task. Ours consistently denotes reference-trajectory tracking, not goal optimization.

TaskBC nominalBC shiftedResidual RLDRDiPacOurs
Toy packing78.328.321.738.328.341.7
T-pushing71.745.063.315.088.355.0
Rope 1clip73.318.321.761.765.061.7
T-rope58.348.360.085.048.353.3
Rope 2clips75.651.239.068.319.568.3

Success rate (%). Ours improves over shifted BC on all five tasks and outperforms DiPac on three, most notably rope 2clips (68.3% versus 19.5%). Other baselines perform better in some tasks; the practical distinction is correction of a frozen policy without online physics-simulator calls or additional policy training.

We also measured cost explicitly. Physical-parameter identification requires 8–240 simulation rollouts, and surrogate-data collection requires 40–64 rollouts for the four tasks with separately reported counts. These are preparatory costs, distinct from online CEM. Online correction latency ranges from 0.21 to 1.85 s, measured on the same hardware for every task. Separately, the surrogate achieves approximately 19–180× faster 60-step dynamics prediction than the full simulator; this prediction speedup is not an end-to-end replanning speedup.

2. Temporal alignment and correction timing

We agree that matching predicted and reference states by absolute time can penalize valid executions that advance more slowly after a dynamics shift. For T-rope and rope 2clips, we therefore use progress-based reference alignment: the current observed task progress determines the corresponding state on the nominal successful reference, rather than the elapsed frame index. This allows correction to target the appropriate manipulation phase even when execution speed changes.

We additionally activate correction when progress stagnates (a progress-stagnation trigger). In rope 2clips, progress-based tracking without this trigger succeeds in 15/41 trials, while adding the trigger increases success to 28/41. This comparison measures the trigger's contribution; a controlled fixed-time-versus-progress-alignment comparison remains a separate question.

3. Surrogate accuracy on CEM-selected corrections

The surrogate directly determines which corrections CEM selects by predicting their future trajectories and evaluating deviation from the reference. We therefore evaluated it during closed-loop simulation: CEM selects a correction, that correction is applied in the physics simulator, and we compare the predicted and realized trajectories. We use each task's actual CEM prediction horizon rather than a uniform rollout length.

TaskCEM horizonPrediction error (mean ± SD)
Toy packing30 steps40.1 ± 36.7 mm
T-pushing60 steps3.84 ± 0.33 mm; 2.27 ± 0.24°
Rope 1clip30 steps7.9 ± 8.6 mm
T-rope30 steps4.36 ± 4.42 mm
Rope 2clips60 steps5.41 ± 0.27 mm

Unlike nominal-trajectory replay, this evaluates the model in the state-action regions reached by actual correction. It directly supports predictive reliability for executed CEM selections, while candidate-ranking agreement across unexecuted alternatives is not measured by these errors alone.

4. Generalization and physical validation

The updated time-index-free BC succeeds on 58.3–78.3% of varied nominal initial configurations. Applying the same BC under shifted dynamics reduces success, and reference-based correction improves success in all five tasks (table above). In addition, correction succeeds on 6 of 10 rope-2clips configurations and 4 of 9 T-pushing trials where nominal BC had failed. This supports recovery beyond a single fixed reference start or predefined execution timeline.

We are currently conducting repeated physical experiments with observation-dependent online correction. We plan to incorporate the results into the final revised manuscript once available.

Thank you for the detailed rebuttal and additional experiments. The updated BC policies, computational cost analysis, progress-based alignment, and surrogate evaluation partially address my concerns.

However, several issues remain. The method still does not outperform the strongest baseline in any task, leaving its practical advantage unclear. The effectiveness of progress-based alignment lacks a controlled comparison, candidate-ranking reliability remains unverified, and online correction has not been validated on hardware.

Overall, I appreciate the authors' efforts, but the remaining concerns are insufficiently resolved to justify a score increase. I maintain my rating of 2.

AuthorsOpenReview ↗

Response to follow-up: alignment, candidate ranking, and adaptation cost

Thank you for the follow-up. We address each remaining concern with new controlled comparisons and measurements in the revised manuscript.

1. Practical advantage over the strongest baseline

We agree that our method does not have the highest success rate in most tasks. It is highest in toy packing (25 vs. 23/60, not significant) and ties DR in single-clip (37/60) and two-clip routing (28/41). We do not claim higher peak success. Its practical advantage lies in two properties.

(a) It is the only method that never falls below the uncorrected policy. The strongest baseline differs by task, and each baseline falls well below the uncorrected policy in at least one task: residual RL by 12.2 percentage points (two-clip routing), DR by 30.0 (T-bar pushing), and DiPac by 31.7 (two-clip routing). Ours improves in all five tasks (by 5.0–43.4 points) and has the highest mean success (56.0%, vs. 53.7% DR, 49.9% DiPac, 41.1% residual RL). When the shift is not known in advance, the best baseline for a task cannot be chosen beforehand, and a correction that does not degrade the frozen policy is the safer choice.

(b) Lower adaptation cost. A new wall-clock table (Appendix A.4), measured on one workstation (i7-10700, one RTX 2080 Ti) with a single worker, shows:

  • preparing our method (identification, surrogate data, and training) takes 1.5–2.6 h, compared with 3.1–15.6 h for residual RL training and 2.4–7.7 h of computation for DR retraining, which also needs 25–45 human demonstrations;
  • correction adds 13 s–2.3 min of computation per execution, compared with 2.7–18.5 min for DiPac. DiPac uses the same identified surrogate in four tasks and is higher in T-bar pushing (53 vs. 33/60);
  • our method needs no policy update, new demonstrations, task reward, or simulator calls during execution.

Correction to our earlier reply: surrogate data uses 60–93 simulator executions per task, not 40–64.

2. Controlled comparison of progress-based vs. fixed-time alignment

In two-clip routing, on the same 41 configurations, we changed only how the reference is indexed: by observed progress (ours) or by elapsed time. The stall trigger, surrogate, and search are the same (Section 6.4).

Two-clip routingSuccess
Fixed-time alignment18/41
Progress alignment (ours)28/41
  • 11 configurations succeed only with progress alignment and 1 only with fixed-time alignment (paired sign test, p ≈ 0.006).
  • When progress first stalls, the time-indexed reference is typically 5–20 s ahead of the execution. Because successful references last only 35–47 s, most later searches track the final released state, and the stall is not resolved.

3. Candidate-ranking reliability

We added a direct test of the decision process (Appendix A.5, new table and figure). At each decision point where a correction was applied, we saved the full simulator state and simulated, for H steps with the policy in closed loop, the selected correction δ*, no offset, the CEM candidates ranked 2, 5, and 10, and a random offset. We then compared each realized cost with the surrogate's prediction.

Results at decision points in contact:

TaskSpearmanδ* better than no offsetRealized / predicted reduction
Toy packing0.379/120.87
T-bar pushing0.6112/140.68
Rope 1clip0.6643/470.91
T-rope pushing0.466/200.77
Rope 2clips0.8350/501.12
  • In T-bar, 1clip, and 2clips, where correction is most active, the ranking correlates with realized costs (0.61–0.83), δ* beats no offset at 86–100% of points, and 68–112% of the predicted reduction is realized.
  • Toy packing has the weakest ranking because its prediction error is large (50 mm after H steps, against a 7.7 mm replay floor).
  • In T-rope pushing, corrections are applied before the stick moves the rope, so candidate costs rarely differ within H steps. This is consistent with its small gain (29 → 32/60).

4. Hardware validation of online correction

We agree that observation-dependent online correction has not yet been validated on physical hardware. Our current physical experiment demonstrates that corrections selected in simulation can be transferred to the real robot and recover a failed manipulation task.

Extending this to closed-loop online correction requires reliable, real-time tracking of the deformable object's state to provide the feedback needed for repeated correction updates. Establishing such a tracking pipeline is beyond the scope of the current study. We therefore limit our hardware claim to the feasibility of transferring simulation-selected corrections, while validating observation-dependent online correction in simulation.

We are currently working toward integrating real-time object tracking with online correction on physical hardware.

We hope these comparisons and measurements resolve the remaining concerns.

Reviewer_94djOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies online recovery of pretrained robotic manipulation policies under changes in object dynamics, such as variations in mass and friction. Rather than retraining or modifying the policy, the authors propose a model-based action correction framework that identifies effective physical parameters from interaction records, trains a lightweight dynamics surrogate under the estimated parameters, and searches over a low-dimensional action correction space during execution. The selected corrections aim to keep the predicted motion close to a successful reference trajectory generated by the same policy under the original dynamics.

The proposed approach is evaluated on rope routing, T-pushing, and toy packing under four simulated dynamics shifts. The results show improvements over the uncorrected policy in all evaluated conditions, with success rates increasing from 0–4% to 52–78% in rope routing and T-pushing, and from 65% to 78% in toy packing. The paper also presents a physical rope-routing experiment in which a correction selected in simulation enables the robot to complete a task that fails with the original policy after a change in rope properties.

The problem is relevant to deploying pretrained manipulation policies in environments where physical properties may differ from those encountered during training. However, the current work leaves important questions regarding the precise problem formulation, the methodological novelty beyond existing model-based action optimization approaches, and the extent to which the experimental evidence supports the claimed effectiveness and generality. In particular, the proposed method achieves partial recovery in several settings but does not consistently outperform alternative adaptation approaches. I therefore lean toward Borderline Reject.

Critical strengths

Practical and relevant problem setting. The paper addresses a meaningful deployment challenge: a pretrained manipulation policy may fail when object dynamics change, even if the task and object geometry remain unchanged. Investigating recovery without updating the original policy is a worthwhile research direction.

A coherent model-based recovery pipeline. The proposed approach combines physical parameter identification, dynamics surrogate learning, and low-dimensional action correction. This provides an interpretable mechanism for modifying policy execution without retraining the policy itself.

Evaluation across multiple manipulation tasks and dynamics shifts. The experiments cover rope routing, T-pushing, and toy packing, with changes in mass and contact friction. The results indicate that execution-time action correction can recover some failures caused by dynamics shifts.

Critical weaknesses

The problem formulation and scope of the proposed contribution need to be clarified. The paper frames the problem as online recovery under dynamics shifts, but the proposed pipeline relies on interaction records for physical parameter identification and on subsequent surrogate-model training before correction selection. The online component primarily consists of searching for action corrections using the trained surrogate. The paper should more precisely distinguish the offline or preparatory adaptation stage from the execution-time recovery stage and clarify what information is assumed to be available after a dynamics shift. More importantly, it remains unclear which specific limitation of existing action optimization or model-based adaptation methods the proposed formulation fundamentally addresses.

The methodological novelty over existing model-based action correction approaches is insufficiently established. The framework combines parameter identification, surrogate-based motion prediction, reference-trajectory tracking, and derivative-free optimization in a restricted action space. These components are individually intuitive, but the paper does not yet convincingly establish the methodological insight that distinguishes this combination from existing residual control, model-based action optimization, and simulation-based planning approaches. The main contribution appears to be a particular integration of these techniques rather than a clearly isolated algorithmic advance. A more direct comparison of the underlying objectives, optimization procedures, and adaptation assumptions is needed to establish the novelty.

The experimental evidence is insufficient to support broad claims about recovery effectiveness. Although the proposed method improves over the uncorrected policy in all evaluated conditions, it does not consistently outperform the adaptation baselines. In several conditions, direct simulator search or residual reinforcement learning achieves 100% success, whereas the proposed method achieves lower success rates. The evaluation also covers only three tasks and four single-parameter dynamics shifts in simulation. The physical validation is limited to one rope-routing scenario, without repeated physical trials or statistical evaluation. Consequently, the experiments demonstrate feasibility in selected settings but do not establish robust recovery across a sufficiently broad range of dynamics changes and manipulation tasks.

The reference-trajectory objective does not reliably align with task success. The primary objective encourages predicted motion to remain close to a successful reference trajectory under the original dynamics. However, trajectory similarity does not necessarily imply task completion under changed dynamics. The paper's own terminal-cost experiment illustrates this limitation: adding a terminal task cost improves T-pushing success from 74% to 100% and toy-packing success from 78% to 87%, but reduces rope-routing success from 52% to 0% under the floor-friction shift. This substantial degradation suggests that the optimization objective does not adequately capture task-phase ordering, contact maintenance, and release conditions. The paper identifies this issue but does not provide a sufficiently general solution.

The reliability of dynamics identification and surrogate predictions requires stronger validation. The identified physical parameters can differ substantially from the shifted ground-truth parameters, indicating that the method relies on effective simulator settings rather than necessarily recovering the true underlying physics. Such effective parameters may still be useful for prediction, but the paper should more clearly establish when the identified simulator provides sufficiently accurate counterfactual predictions for action selection. In particular, the consequences of surrogate errors in candidate correction regions are not fully characterized. The current results do not establish how reliably the method avoids corrections that appear beneficial under the surrogate but fail in the actual environment.

Questions for authors

Problem formulation and assumptions: What precisely distinguishes the proposed online recovery problem from existing model-based action optimization and residual-control formulations? Which information is assumed to be available after a dynamics shift, and how much interaction data is required before online correction can begin?

Methodological novelty: Beyond combining dynamics identification, surrogate prediction, reference-trajectory tracking, and low-dimensional optimization, what is the central methodological insight that cannot be readily obtained from existing action correction or model-based planning methods? Could the authors isolate this contribution through a more direct comparison or targeted ablation?

Objective design: Why is tracking the original successful trajectory the primary optimization objective instead of directly optimizing a task-relevant cost? Given the substantial performance degradation of the terminal-cost variant in rope routing, how can the method represent task-phase ordering, contact maintenance, and release constraints without introducing task-specific objectives?

1 comment
AuthorsOpenReview ↗

Methodological contribution and recovery evaluation

Thank you for the detailed feedback. We clarified the formulation and expanded the evaluations.

Problem formulation and contribution

Our framework separates preparatory adaptation from online correction. Recorded actions and observations support effective-parameter identification and surrogate training; execution then optimizes low-dimensional offsets to a frozen BC policy using current observations and a reference trajectory, without online simulator calls. Identification uses 8–240 simulated rollouts; surrogate-data collection uses 40–64 in four tasks. These are simulation costs, not physical interaction counts.

Our contribution is reference-guided recovery of a frozen, observation-dependent policy, rather than a new generic optimizer. Successful object, gripper, and contact trajectories provide a correction objective without manually designed terminal costs; the surrogate enables repeated corrections without online simulation.

Updated BC and complete re-evaluation

The submitted BC used a time index and fixed initial configurations. We retrained BC on varied initial object/robot poses, removed time inputs entirely, and retained observation histories. All objective ablations and adaptation baselines were re-evaluated from these updated policies with matched task-specific initial conditions and success criteria. Ours always means trajectory tracking. Four tasks have 60 trials; two-clip routing has 41.

TaskUpdated BCResidual RLDomain Rand.DiPac replanningOurs
Toy packing28.3%21.7%38.3%28.3%41.7%
T-pushing45.0%63.3%15.0%88.3%55.0%
Rope 1clip18.3%21.7%61.7%65.0%61.7%
T-rope pushing48.3%60.0%85.0%48.3%53.3%
Rope 2clips51.2%39.0%68.3%19.5%68.3%

Ours improves over uncorrected BC in all five tasks, leads on toy packing, and ties DR on two-clip routing, without updating BC weights.

Why track the successful trajectory?

The updated objective ablation compares terminal-goal optimization with trajectory tracking:

TaskTerminal goalOurs (trajectory)
Toy packing53.3%41.7%
T-pushing98.3%55.0%
Rope 1clip36.7%61.7%
T-rope pushing40.0%53.3%
Rope 2clips4.9%68.3%

Goal optimization wins on toy packing and T-pushing; tracking wins on all three rope tasks by preserving intermediate object/gripper/contact behavior. For T-rope and two-clip routing, reference alignment uses observed task progress, not absolute time. A progress-stagnation trigger increases two-clip success from 15/41 to 28/41; this isolates the trigger within progress-aligned tracking.

Does physical identification help?

With correction settings fixed, we compared nominal- and identified-physics surrogates:

TaskNominal surrogateIdentified surrogate
T-bar pushing27/6033/60
Rope 1clip62/12073/120
Rope 2clips63/12168/121
Toy packing20/6026/60
T-rope pushing31/6031/60

Identification improves single-clip routing (paired p = 0.019) on both sets. Correction drives much of the recovery; estimated effective dynamics offer further gains under mismatch without necessarily matching true parameters.

Surrogate accuracy in the actual correction regime

We measured errors for CEM-selected corrections executed in simulation, using each task’s actual planning horizon:

TaskCEM horizonPrediction error (mean ± std.)
Toy packing30 steps40.1 ± 36.7 mm
T-pushing position / orientation60 steps3.84 ± 0.33 mm / 2.27 ± 0.24°
Rope 1clip30 steps7.9 ± 8.6 mm
T-rope pushing30 steps4.36 ± 4.42 mm
Rope 2clips60 steps5.41 ± 0.27 mm

These errors validate prediction along executed corrections; ranking unexecuted candidates remains a separate question.

Reviewer_7rnCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper recovers a frozen manipulation policy after changes in mass or friction. It fits an effective simulator, trains a dynamics surrogate, and repeatedly adjusts actions to track a successful reference trajectory. Evaluation covers four simulated shifts and a separate physical rope experiment.

Critical strengths
  • The method substantially recovers policies that otherwise fail: success rises from 0–4% to 52–78% on rope routing and T-pushing, without updating policy weights.
  • The objective ablations are informative. Adding a terminal task cost helps some tasks but hurts rope routing, showing that the choice of correction objective matters.
Critical weaknesses
  • The benefit over existing model-based replanning is not isolated. DINO-WM already uses MPC, while the direct-search baseline here fixes its correction schedule before execution. Could the authors compare against a replanning baseline with matched state information, objective, and search budget?
  • The practical cost remains unclear. Adaptation requires simulator fitting and surrogate training, and the physical experiment does not test the full observation-dependent correction loop repeatedly. Please report adaptation time, correction latency, and repeated hardware results.
Questions for authors

See weaknesses

1 comment
AuthorsOpenReview ↗

Online replanning and computational efficiency

We thank the reviewer for highlighting the recovery results and the informative objective ablations. We have strengthened the online-replanning comparison and measured the computational costs of the full correction procedure.

1. Updated BC policies and evaluation protocol

The BC policies in the submission used a time-index input and were trained around a fixed initial configuration. To assess recovery across different initial conditions, we retrained the BC policies using demonstrations collected from varying initial object and robot poses and removed the time-index input entirely. The updated policies predict actions from observation histories, while their weights remain frozen during dynamics adaptation and correction.

Importantly, we re-evaluated every objective ablation and adaptation baseline starting from these updated BC policies, using common initial configurations and success criteria within each task. Four tasks were evaluated over 20 initial configurations with three repetitions (60 trials), and two-clip rope routing over 41 configurations (41 trials). We also included two Real2Sim-based simulation tasks and revised two dynamics shifts. Hence, these are new, internally consistent evaluations rather than a numerical reproduction of the original submission.

2. Comparison with online model-based replanning

We agree that the submitted fixed-sequence direct-search baseline did not adequately represent observation-dependent replanning. We therefore added an online simulation-based baseline based on Differentiable Particles for General-Purpose Deformable Object Manipulation (DiPac; Chen et al., 2024), and evaluated it with the updated BC policies.

TaskDiPac-based online replanningOurs (trajectory tracking)
Toy packing28.3% (17/60)41.7% (25/60)
T-pushing88.3% (53/60)55.0% (33/60)
Rope routing, 1 clip65.0% (39/60)61.7% (37/60)
T-rope pushing48.3% (29/60)53.3% (32/60)
Rope routing, 2 clips19.5% (8/41)68.3% (28/41)

Ours achieves higher success on three of five tasks, with a particularly large gain on two-clip routing (68.3% vs. 19.5%). DiPac performs better on T-pushing and slightly better on single-clip routing. This addition addresses the absence of an online replanning baseline in the submission.

The approaches are evaluated under common task conditions, although their state representations, objectives, planning horizons, and search budgets are not fully matched. We therefore interpret this as a system-level comparison rather than an isolated test of surrogate modeling. The key computational distinction is that DiPac-based replanning evaluates actions through physics simulation online, whereas Ours optimizes corrections using its learned surrogate with zero online physics-simulator calls.

3. Adaptation costs and measured correction latency

We separately report preparatory adaptation requirements and online correction costs.

ComponentOurs
Effective-parameter identification8–240 simulated rollouts, depending on task
Surrogate-data collection40–64 rollouts in the four tasks with separately reported counts
Online physics-simulator calls0
Online correction computation0.21–1.85 s, depending on task
60-step dynamics predictionApproximately 19–180× faster than full simulation in the reported prediction benchmarks

All task-level online correction times were measured on the same hardware, allowing consistent comparison of correction computation across tasks:

TaskCorrection computationNominal replanning interval
T-pushing (T-bar)0.21 s0.33 s
T-rope pushing0.59 s0.80 s
Toy packing0.34 s0.33 s
Rope routing, 1 clip0.48 s0.33 s
Rope routing, 2 clips1.85 s0.33 s

Parameter identification and surrogate training occur before execution. During execution, CEM evaluates corrections using the surrogate, without physics-simulator calls. The 19–180× figures refer specifically to dynamics-prediction benchmarks, not end-to-end CEM speedups; unlike the online correction timings, those prediction benchmarks used task-dependent simulator and hardware configurations. We will present these cost categories separately in the revision.

4. Physical experiments

The submitted physical experiment demonstrated transfer of a simulation-selected correction. We are currently conducting repeated physical experiments with observation-dependent online action correction to assess the complete execution-time procedure. We plan to incorporate the results into the final revised manuscript once available.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of revisions

We thank all reviewers again. Below are the main changes, each with the reviewers whose comments it addresses.

  1. Updated policies and complete re-evaluation (7rnC, 94dj, AxWW) The BC policies are retrained from demonstrations with varied initial configurations, without a time input (Eq. 1). All ablations and baselines are re-evaluated on common initial configurations: 60 executions per task (20 configurations × 3) and 41 configurations in two-clip routing, including configurations where the policy fails even under the reference physics (Sections 3.1, 5.1, 5.3; appendix).

  2. Broader evaluation (94dj, AxWW) Five tasks: toy packing, T-bar pushing, and single-clip routing in PhysTwin-based scenes, and T-rope pushing and two-clip routing in MuJoCo scenes built from real recordings. The shifts change gripper, pusher, floor, and table friction, and rope bending stiffness and damping (Section 5.1).

  3. Online replanning baseline (7rnC) A DiPac-style baseline re-optimizes the full action sequence through an identified-physics surrogate at every correction interval, presented as a system-level comparison. The direct-search baseline is removed (Sections 2.2, 5.2, 6.2).

  4. Results relative to the baselines (94dj, AxWW) Online correction improves over the uncorrected policy in all five tasks (28.3→41.7%, 45.0→55.0%, 18.3→61.7%, 48.3→53.3%, 51.2→68.3%). It is best in toy packing and ties DR in single- and two-clip routing; DiPac, DR, or residual RL is higher in T-bar pushing, single-clip routing, and T-rope pushing. We state this and describe baseline failure cases (Section 6.2, Conclusion).

  5. Problem formulation and adaptation cost (7rnC, 94dj, AxWW) The introduction states the central idea: the motion of the frozen policy under the reference physics serves as the correction objective, preserving the order of grasping, contact, and release without a hand-designed task cost. We separate preparation (identification and surrogate training) from execution, in which no simulator is called, and state the assumptions after a shift; identification searches only the changed parameter (Sections 1, 3.2, 4.2). The appendix reports identification and surrogate-data counts, correction settings, one-correction time (0.21–1.85 s), and surrogate speed (19–180× faster than simulation over 60 steps). A new wall-clock table (one workstation, single worker) shows 1.5–2.6 h of preparation for our method, vs. 3.1–15.6 h for residual RL and 2.4–7.7 h of computation plus human demonstrations for DR, and 13 s–2.3 min of correction per execution vs. 2.7–18.5 min for DiPac (Section 6.2; Appendix A.4).

  6. Temporal alignment (AxWW) Alignment is task-specific: the frame index is shifted at the first grasp contact in toy packing and single-clip routing, and the reference follows observed progress in T-rope pushing and two-clip routing, where correction starts only when progress stalls (15/41→28/41 in two-clip routing; Sections 4.4, 6.4). With everything else fixed, indexing the reference by elapsed time instead of progress gives 18/41 vs. 28/41 (paired sign test p≈0.006), because the time-indexed reference runs 5–20 s ahead at the first stall (Section 6.4).

  7. Objective design (94dj, AxWW) The ablation compares goal, goal-plus-trajectory, and trajectory objectives in all five tasks. The goal objective is better in T-bar pushing and toy packing; trajectory tracking is better in the three rope tasks, where success depends on the order of grasp and release (Section 6.4).

  8. Identification (94dj) Identified coefficients are close to the shifted values, and held-out replay errors under the identified parameters are comparable to those under the shifted parameters in T-bar pushing, single-clip routing, and T-rope pushing; toy packing replays branch between holding and slipping. Replayed shapes are shown for all tasks (Section 6.3; appendix). Compared with a surrogate trained under the reference physics, the identified-physics surrogate raises single-clip routing from 62 to 73 of 120 executions (p≈0.019), the only significant difference (Section 6.3; appendix).

  9. Correction behavior (AxWW) Representative executions and the correction-signal figure (four tasks) are updated (Section 6.1).

  10. Related work (7rnC) The description of DINO-WM is corrected (Section 2.1).

  11. Surrogate accuracy at applied corrections (94dj, AxWW) At decision points where a correction was applied, we restore the simulator state and simulate the selected correction, no offset, and other CEM candidates. While the object is held or pushed, the selected correction beats no offset at 50/50 points in two-clip routing, 43/47 in single-clip routing, 12/14 in T-bar pushing, and 9/12 in toy packing, realizing 0.68–1.12 of the predicted cost reduction; T-rope pushing is weaker (6/20). A new table and figure report prediction errors, rank correlations, and candidate motion (Sections 6.3, 7; Appendix A.5).

AuthorsOpenReview ↗

Update for All Reviewers: Repeated Physical Experiments with Online Correction

We have completed repeated physical experiments with online, observation-dependent correction, addressing the evaluation discussed in our earlier responses (7rnC and AxWW). These results are now included in the revised manuscript (Section 6.5), with two supplementary videos demonstrating online correction on the physical robot for both tasks. The new experiments replace the previous single open-loop execution.

Setup

  • We evaluate two tasks on a UR5e robot under changed physical properties: two-clip routing with a stiffer braided rope and T-rope pushing on a higher-friction silicone mat.
  • The frozen policies, references, surrogate, and correction procedure remain unchanged from Section 4. Object states are tracked online using three RGB-D cameras at approximately 10 Hz, and physical parameters are estimated once per changed condition and held fixed during execution.
  • Each condition is evaluated from five distinct initial configurations selected from the simulated evaluation set. Success is assessed visually.

Results

TaskOriginal property, uncorrectedChanged property, uncorrectedChanged property, online correction
Two-clip routing (braided rope)3/5 (60%)3/5 (60%)5/5 (100%)
T-rope pushing (silicone mat)0/5 (0%)0/5 (0%)3/5 (60%)
  • Two-clip routing: Online correction improves success from 60% to 100%. Correction begins after grasping and is disabled when reference progress stalls for ten consecutive decisions, after which the frozen policy completes clip passage. Of 707 decisions, 255 apply a nonzero offset, mostly near the 20 mm bound.
  • T-rope pushing: Online correction improves success from 0% to 60%. The uncorrected policy fails even on the original mat despite succeeding in simulation, indicating a sim-to-real gap. Tracking the successful simulated reference enables task completion without policy retraining, demonstrating the effectiveness of action correction beyond adaptation to friction changes. The three successful executions finish within 35 mm and 6° of the target T pose, although none meets the 12 mm shape criterion used in simulation.
  • Each correction search introduces a 0.3–0.5 s pause every 0.8 s.

Changes in the Manuscript and Supplementary Material

  • Section 6.5 now presents the repeated physical experiments, including a new figure comparing the original rope, the braided rope without correction, correction offsets over time, and the braided rope with correction.
  • A new supplementary figure presents T-rope pushing with and without correction.
  • Two supplementary videos provide visual demonstrations of the physical robot experiments: one for two-clip routing with the braided rope and one for T-rope pushing.
  • The abstract, introduction, and conclusion have been updated to reflect these results. The limitations discuss the evaluation scope and correction latency.
  • The previous single open-loop execution and its supplementary material have been replaced by the new online experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.