acceptodds
Withdrawn from ICLR 2027

Is Online Interaction Necessary for Recovery? A Minimalist Approach to Robust Planning via Perturbation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Behavior cloning (BC) is vulnerable to covariate shift during closed-loop execution, where small prediction or execution errors can drive the robot toward states poorly covered by the demonstration data. We focus on action-sequence planning, where a policy predicts a finite-horizon sequence of actions as a reference trajectory for robot execution. Existing approaches to covariate shift often rely on collecting additional corrective demonstrations, requiring further environment interaction and access to an expert or reference policy. We propose Perturbation-Augmented Recovery Supervision (PARS), a simple training approach for improving recovery using only existing demonstrations. PARS perturbs the robot's proprioceptive state and anchors only the later portion of the predicted trajectory to the original demonstration, leaving the earlier portion free to generate corrective motion and recover toward the demonstrated behavior within the planning horizon. Unlike conventional input-noise augmentation, which preserves the original supervision target over the entire prediction horizon, PARS explicitly provides trajectory-level supervision for recovery from perturbed states. PARS requires neither additional environment interaction nor expert queries and can be instantiated across different action-sequence policy classes with only minor modifications to their BC objectives. Experiments on 51 RLBench manipulation tasks with flow-based, transformer-based, and diffusion-based policies demonstrate that PARS improves robustness to covariate shift during closed-loop execution.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Behavior cloning, Planning, Robot Manipulation, Recovery, Covariate shift

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wsomOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Perturbation-Augmented Recovery Supervision (PARS) for action-sequence behavior cloning. It perturbs proprioceptive observations and supervises only the predicted trajectory suffix, leaving the prefix flexible for corrective motion. The method uses existing demonstrations without collecting corrective trajectories or querying an expert during training. Across 51 RLBench tasks and six policy variants, the reported mean success-rate gains range from approximately 6.7 to 11.8 percentage points.

The approach is simple, well motivated, and empirically promising, but the current evidence does not sufficiently establish robustness to model selection or confirm the proposed recovery mechanism. The most valuable additions would be independently selected multi-seed results, actual recovery rollouts, and targeted horizon ablations.

Critical strengths

The method is intuitive and addresses a practically important problem. Allowing an initial correction before requiring agreement with demonstrated behavior is well motivated. Evaluation across transformer, flow-based, and diffusion-based policies supports the potential breadth of the approach. The trajectory and kinematic analyses provide useful context beyond success rates, and the manuscript appropriately acknowledges several limitations. Overall, the reported improvements make PARS a promising training strategy.

Critical weaknesses
  1. Robustness of the reported improvements. Table 1 reports the best augmentation setting for each task, while Appendix D lists a single training seed. The separation between hyperparameter selection and final testing is not clearly described, and the comparability of tuning budgets remains unclear. This leaves uncertainty about the influence of selection and training randomness. Independent validation, multiple training seeds, and uncertainty estimates would strengthen the results. A shared augmentation configuration would also help assess reliance on task-specific tuning.

  2. Evidence for actual recovery. Figure 2 evaluates proximity of predicted joint configurations to demonstrations. Comparing policies at the same off-nominal observations is useful, but this proxy does not establish successful recovery during execution. Contact dynamics, object state, and tracking errors can affect whether the motion restores task progress. Executing the policies from matched off-nominal simulator states and reporting subsequent success and recovery time would more directly support the central claim.

  3. Missing ablations and limited scope. The experiments fix a 10-step free prefix within a 20-step prediction horizon and execute the full chunk before observing again. Varying prefix length and replanning frequency would clarify when suffix anchoring helps. Comparisons with alternative supervision schedules should use comparable losses and tuning budgets. The theory offers conditional local insight, while the experiments use low-dimensional simulated observations; broader claims about recovery and applicability should remain proportionate to this evidence.

Questions for authors
  1. Hyperparameter selection: How were the task-specific augmentation settings selected? Were validation episodes separate from the final evaluation, and were comparable tuning budgets used for PARS and noise augmentation?

  2. Robustness of the results: The manuscript reports a single training seed. Are additional results available to assess whether the improvements are consistent across seeds or less dependent on task-specific tuning?

  3. Evidence of recovery: Beyond the predicted-trajectory analysis in Figure 2, is there evidence that PARS improves recovery during actual execution? For example, were the compared policies executed from the same off-nominal states to assess subsequent task success?

Reviewer_XEtmOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Perturbation-Augmented Recovery Supervision (PARS), a training augmentation method intended to improve recovery from covariate shift in action-sequence policies. PARS perturbs the robot’s proprioceptive state and supervises only the later portion of the predicted trajectory against the demonstrated continuation, while retaining full-sequence behavior cloning on unperturbed examples. In the experiments, policies predict 20-step action sequences, with augmented supervision applied to the final 10 steps. For diffusion and conventional flow policies, sampled sequences are refined using the suffix loss and then used as training targets. Evaluation covers 51 simulated RLBench tasks with low-dimensional state observations and several policy classes. The paper reports improved task success over unaugmented policies and conventional noise augmentation, without collecting additional corrective demonstrations.

Critical strengths
  • Broad evaluation across policy classes. Experiments cover 51 simulated tasks and multiple flow, transformer, and diffusion policies, with reported mean success improvements across all evaluated methods.
  • Relevant augmentation baseline. Comparison with conventional full-target noise augmentation helps assess the benefit of supervising only the trajectory suffix.
  • Reuse of existing demonstrations. The method constructs additional training supervision without collecting corrective demonstrations or querying an expert at perturbed states.
  • Analysis beyond task success. Predicted recovery-distance curves, speed and acceleration measurements, and conditional theoretical analysis provide additional insight into the proposed mechanism.
Critical weaknesses
  • The recovery mechanism is insufficiently established. The objective constrains later trajectory positions but does not explicitly distinguish useful early recovery from delayed or physically unsuitable corrections. Leaving the prefix unconstrained does not itself establish that effective recovery will emerge.
  • The evaluation has limited scope. All experiments use low-dimensional state observations in simulation, with one variation per task. Applicability to visual policies and physically consistent disturbances remains untested.
  • Recovery evidence is indirect. The recovery and kinematic analyses examine predicted trajectories rather than their executed outcomes. Proximity to demonstrated joint configurations does not necessarily indicate correct task progress or successful recovery.
  • The central temporal choice lacks ablation. The method fixes a 10-step free prefix within a 20-step horizon without systematically testing other splits or execution horizons. The importance of this choice is therefore unclear.
  • Alternative explanations are not sufficiently isolated. The experiments do not clearly distinguish recovery learning from suffix invariance or regularization, nor establish whether similar gains could follow from other training schemes addressing policy-induced deviations.
  • Robustness to experimental choices is unclear. Results use the best augmentation settings selected separately for each task, while the reported training setup specifies a single random seed. The selection protocol and variability across training runs require stronger validation.
Questions for authors
  1. Recovery mechanism. What evidence shows that suffix supervision teaches effective recovery during the first 10 steps, rather than mainly making later predictions less sensitive to state perturbations? Can these two explanations be separated experimentally?

  2. Executed recovery. Can the recovery analysis be repeated by executing the predicted chunks from the same off-nominal states? Actual joint trajectories, task progress, and failures during the prefix would provide stronger evidence than predicted distance to demonstrated configurations alone.

  3. Observation consistency. Under what assumptions does changing the robot state while keeping environmental observations fixed produce a valid training example? How is this handled when the robot holds an object or is in contact with the environment, and what would be required for visual observations?

  4. Temporal design and alternatives. How does performance vary with the free-prefix length and replanning frequency? Would a simple, explicitly constructed recovery prefix—such as gradually removing the initial offset—perform similarly to leaving the prefix without direct targets?

  5. Training comparisons. Can the method be compared with another training scheme that exposes the policy to its own generated histories or deviations, using a compatible policy formulation and comparable budget? This would help distinguish the benefit of suffix anchoring from broader reductions in training–execution mismatch.

  6. Selection and reproducibility. What data and evaluation rollouts are used to select the best augmentation settings for each task? Are they separate from the final evaluation episodes, how much environment interaction does tuning require, and do the improvements persist across multiple training seeds?

  7. Real-robot validation. Can the method be evaluated on a physical robot under controlled disturbances? Such experiments would help establish whether suffix anchoring produces executable recovery despite tracking errors, contact changes, and observation inconsistency—effects not directly validated by the current state-based simulation experiments.

Reviewer_59z9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how to train action-sequence policies to recover from execution errors using only existing demonstrations. The proposed Perturbation-Augmented Recovery Supervision (PARS) algorithm perturbs proprioceptive observations and supervises the predicted trajectory's later portion, leaving its initial portion free to generate corrective motion. The theoretical analysis characterizes the resulting local sensitivity regularization and gives sufficient conditions for the initial motion to remain close to demonstrations and satisfy the demonstration planner's joint-space speed and acceleration limits under stated local assumptions. Experiments on 51 RLBench tasks show improved success rates across six policy variants, with predicted trajectories exhibiting better recovery or reduced divergence from demonstrated behavior.

Critical strengths

Improving recovery in imitation learning without collecting additional corrective demonstrations is a valuable research problem.

Critical weaknesses
  • The method does not verify whether a perturbed state admits a feasible recovery trajectory to the demonstrated suffix. For example, when obstacles are near the robot,such synthesized training pairs may provide infeasible recovery targets, potentially encouraging the policy to generate trajectories that cause collisions. This raises concerns about its applicability across different types of manipulation tasks and whether the augmentation could sometimes degrade performance.
  • The experiments do not include visual input settings. Real-world manipulation policies commonly rely on vision, but it is unclear how images could be modified consistently with perturbed robot states, especially for wrist cameras, where small end-effector movements can substantially change the field of view.
  • This method has not been validated through real world experiments.
Questions for authors

Table 1 reports the best augmentation setting selected separately for each task. This raises concerns about how much the reported gains depend on task-specific hyperparameter tuning. Could the authors report results using a single augmentation setting shared across all tasks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.