- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies recovery of a frozen manipulation policy under dynamics shifts. It identifies effective simulator parameters, trains a dynamics surrogate, and searches for low-dimensional action corrections whose predicted trajectories track a successful reference execution. Experiments on rope routing, T-pushing, and toy packing report improved success over the uncorrected policy, complemented by a physical rope demonstration.
Critical strengths
-
Relevant and well-defined adaptation setting. The paper addresses failures caused by changed dynamics while retaining the pretrained policy. Restricting adaptation to low-dimensional action corrections provides a concrete alternative to policy retraining.
-
Substantial recovery in the evaluated settings. Success increases from 0% to 78% under the rope mass shift and from 0% to 74% in T-pushing, with improvements across all four conditions. The physical rope demonstration provides preliminary evidence that a simulation-selected correction can transfer to hardware.
-
A concrete design for repeated correction. The surrogate enables candidate evaluation without simulator calls during execution. Observation-dependent updates, ensemble disagreement penalties, and correction regularization form a coherent mechanism for adjusting the frozen policy's actions.
-
Transparent reporting and useful failure analysis. The paper reports stronger baseline results and negative terminal-cost results rather than presenting uniformly favorable comparisons. The gripper diagnostics connect rope-routing failures to premature release, while the discussion distinguishes effective simulator parameters from true physical properties.
Critical weaknesses
1. The practical advantage over the best baselines remains unclear.
The proposed method does not outperform the best baseline in any of the four simulated conditions; the terminal-cost variant only matches the best baseline on T-pushing. Tables A1–A2 report adaptation execution counts, but not total adaptation time or measured online correction latency. Consequently, it remains unclear what efficiency benefit compensates for the success-rate gap.
Please quantify this success–cost trade-off and further analyze baseline failures, particularly residual RL under increased floor friction and direct search in toy packing. Section 6.2 identifies a simulation-to-shifted-physics mismatch for the latter, but does not isolate its cause. These analyses would clarify when the proposed method is preferable.
2. Fixed temporal alignment may conflate delayed progress with incorrect behavior.
Equation 6 compares predicted and reference states at matching time indices. Both dynamics shifts and execution variability can change task progression, so an appropriate but slower execution may be penalized or encouraged toward a premature task phase. No targeted temporal-alignment robustness study is reported. Please evaluate temporal offsets and execution-speed variation, and compare fixed alignment with monotonic progress-based alignment.
3. Discrimination between candidate trajectory outcomes is insufficiently validated.
In toy packing, direct search passes selection under estimated physics but achieves 0% success under shifted physics (Section 6.2), showing that simulated success need not transfer. This does not establish a failure of the proposed surrogate, but motivates testing whether its candidate rankings predict realized outcomes. Section 6.3 validates simulator motion fitting rather than this decision process. Please visualize predicted and realized trajectories for successful and failed candidates, and quantify candidate-ranking agreement or separation of predicted scores by realized outcome.
4. Generalization beyond the reference configuration remains unverified.
The method restores behavior around a previously successful trajectory, with evaluation restricted to fixed scene geometry and task goals. Although consistent with the stated scope, this limits the demonstrated practical applicability. Modest changes in initial object positions or obstacle layouts would help characterize when reference tracking remains useful.
The physical experiment transfers a simulation-selected correction rather than validating observation-dependent online updates. As acknowledged in Section 7, the central online mechanism therefore remains unverified on hardware. Repeated physical trials with online updates would strengthen the practical evidence.
Questions for authors
Mentioned in Weaknesses
3 comments
Thank you for the detailed rebuttal and additional experiments. The updated BC policies, computational cost analysis, progress-based alignment, and surrogate evaluation partially address my concerns.
However, several issues remain. The method still does not outperform the strongest baseline in any task, leaving its practical advantage unclear. The effectiveness of progress-based alignment lacks a controlled comparison, candidate-ranking reliability remains unverified, and online correction has not been validated on hardware.
Overall, I appreciate the authors' efforts, but the remaining concerns are insufficiently resolved to justify a score increase. I maintain my rating of 2.
Response to follow-up: alignment, candidate ranking, and adaptation cost
Thank you for the follow-up. We address each remaining concern with new controlled comparisons and measurements in the revised manuscript.
1. Practical advantage over the strongest baseline
We agree that our method does not have the highest success rate in most tasks. It is highest in toy packing (25 vs. 23/60, not significant) and ties DR in single-clip (37/60) and two-clip routing (28/41). We do not claim higher peak success. Its practical advantage lies in two properties.
(a) It is the only method that never falls below the uncorrected policy. The strongest baseline differs by task, and each baseline falls well below the uncorrected policy in at least one task: residual RL by 12.2 percentage points (two-clip routing), DR by 30.0 (T-bar pushing), and DiPac by 31.7 (two-clip routing). Ours improves in all five tasks (by 5.0–43.4 points) and has the highest mean success (56.0%, vs. 53.7% DR, 49.9% DiPac, 41.1% residual RL). When the shift is not known in advance, the best baseline for a task cannot be chosen beforehand, and a correction that does not degrade the frozen policy is the safer choice.
(b) Lower adaptation cost. A new wall-clock table (Appendix A.4), measured on one workstation (i7-10700, one RTX 2080 Ti) with a single worker, shows:
- preparing our method (identification, surrogate data, and training) takes 1.5–2.6 h, compared with 3.1–15.6 h for residual RL training and 2.4–7.7 h of computation for DR retraining, which also needs 25–45 human demonstrations;
- correction adds 13 s–2.3 min of computation per execution, compared with 2.7–18.5 min for DiPac. DiPac uses the same identified surrogate in four tasks and is higher in T-bar pushing (53 vs. 33/60);
- our method needs no policy update, new demonstrations, task reward, or simulator calls during execution.
Correction to our earlier reply: surrogate data uses 60–93 simulator executions per task, not 40–64.
2. Controlled comparison of progress-based vs. fixed-time alignment
In two-clip routing, on the same 41 configurations, we changed only how the reference is indexed: by observed progress (ours) or by elapsed time. The stall trigger, surrogate, and search are the same (Section 6.4).
| Two-clip routing | Success |
|---|---|
| Fixed-time alignment | 18/41 |
| Progress alignment (ours) | 28/41 |
- 11 configurations succeed only with progress alignment and 1 only with fixed-time alignment (paired sign test, p ≈ 0.006).
- When progress first stalls, the time-indexed reference is typically 5–20 s ahead of the execution. Because successful references last only 35–47 s, most later searches track the final released state, and the stall is not resolved.
3. Candidate-ranking reliability
We added a direct test of the decision process (Appendix A.5, new table and figure). At each decision point where a correction was applied, we saved the full simulator state and simulated, for H steps with the policy in closed loop, the selected correction δ*, no offset, the CEM candidates ranked 2, 5, and 10, and a random offset. We then compared each realized cost with the surrogate's prediction.
Results at decision points in contact:
| Task | Spearman | δ* better than no offset | Realized / predicted reduction |
|---|---|---|---|
| Toy packing | 0.37 | 9/12 | 0.87 |
| T-bar pushing | 0.61 | 12/14 | 0.68 |
| Rope 1clip | 0.66 | 43/47 | 0.91 |
| T-rope pushing | 0.46 | 6/20 | 0.77 |
| Rope 2clips | 0.83 | 50/50 | 1.12 |
- In T-bar, 1clip, and 2clips, where correction is most active, the ranking correlates with realized costs (0.61–0.83), δ* beats no offset at 86–100% of points, and 68–112% of the predicted reduction is realized.
- Toy packing has the weakest ranking because its prediction error is large (50 mm after H steps, against a 7.7 mm replay floor).
- In T-rope pushing, corrections are applied before the stick moves the rope, so candidate costs rarely differ within H steps. This is consistent with its small gain (29 → 32/60).
4. Hardware validation of online correction
We agree that observation-dependent online correction has not yet been validated on physical hardware. Our current physical experiment demonstrates that corrections selected in simulation can be transferred to the real robot and recover a failed manipulation task.
Extending this to closed-loop online correction requires reliable, real-time tracking of the deformable object's state to provide the feedback needed for repeated correction updates. Establishing such a tracking pipeline is beyond the scope of the current study. We therefore limit our hardware claim to the feasibility of transferring simulation-selected corrections, while validating observation-dependent online correction in simulation.
We are currently working toward integrating real-time object tracking with online correction on physical hardware.
We hope these comparisons and measurements resolve the remaining concerns.
Progress-aware correction, computational cost, and generalization
Thank you for the constructive comments. We have updated the BC policies and re-evaluated all ablations and baselines, with additional analyses of computational cost, temporal alignment, and surrogate accuracy during correction.
1. Recovery performance and computational cost
The submitted BC policies used a time index and were trained for fixed initial configurations. We retrained BC with demonstrations from varied initial object/robot poses, completely removed the time-index input, and retained observation-history inputs. All objective ablations and adaptation baselines were re-evaluated using these updated BC policies and the same evaluation conditions within each task. Ours consistently denotes reference-trajectory tracking, not goal optimization.
Success rate (%). Ours improves over shifted BC on all five tasks and outperforms DiPac on three, most notably rope 2clips (68.3% versus 19.5%). Other baselines perform better in some tasks; the practical distinction is correction of a frozen policy without online physics-simulator calls or additional policy training.
We also measured cost explicitly. Physical-parameter identification requires 8–240 simulation rollouts, and surrogate-data collection requires 40–64 rollouts for the four tasks with separately reported counts. These are preparatory costs, distinct from online CEM. Online correction latency ranges from 0.21 to 1.85 s, measured on the same hardware for every task. Separately, the surrogate achieves approximately 19–180× faster 60-step dynamics prediction than the full simulator; this prediction speedup is not an end-to-end replanning speedup.
2. Temporal alignment and correction timing
We agree that matching predicted and reference states by absolute time can penalize valid executions that advance more slowly after a dynamics shift. For T-rope and rope 2clips, we therefore use progress-based reference alignment: the current observed task progress determines the corresponding state on the nominal successful reference, rather than the elapsed frame index. This allows correction to target the appropriate manipulation phase even when execution speed changes.
We additionally activate correction when progress stagnates (a progress-stagnation trigger). In rope 2clips, progress-based tracking without this trigger succeeds in 15/41 trials, while adding the trigger increases success to 28/41. This comparison measures the trigger's contribution; a controlled fixed-time-versus-progress-alignment comparison remains a separate question.
3. Surrogate accuracy on CEM-selected corrections
The surrogate directly determines which corrections CEM selects by predicting their future trajectories and evaluating deviation from the reference. We therefore evaluated it during closed-loop simulation: CEM selects a correction, that correction is applied in the physics simulator, and we compare the predicted and realized trajectories. We use each task's actual CEM prediction horizon rather than a uniform rollout length.
Unlike nominal-trajectory replay, this evaluates the model in the state-action regions reached by actual correction. It directly supports predictive reliability for executed CEM selections, while candidate-ranking agreement across unexecuted alternatives is not measured by these errors alone.
4. Generalization and physical validation
The updated time-index-free BC succeeds on 58.3–78.3% of varied nominal initial configurations. Applying the same BC under shifted dynamics reduces success, and reference-based correction improves success in all five tasks (table above). In addition, correction succeeds on 6 of 10 rope-2clips configurations and 4 of 9 T-pushing trials where nominal BC had failed. This supports recovery beyond a single fixed reference start or predefined execution timeline.
We are currently conducting repeated physical experiments with observation-dependent online correction. We plan to incorporate the results into the final revised manuscript once available.