- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Perturbation-Augmented Recovery Supervision (PARS) for action-sequence behavior cloning. It perturbs proprioceptive observations and supervises only the predicted trajectory suffix, leaving the prefix flexible for corrective motion. The method uses existing demonstrations without collecting corrective trajectories or querying an expert during training. Across 51 RLBench tasks and six policy variants, the reported mean success-rate gains range from approximately 6.7 to 11.8 percentage points.
The approach is simple, well motivated, and empirically promising, but the current evidence does not sufficiently establish robustness to model selection or confirm the proposed recovery mechanism. The most valuable additions would be independently selected multi-seed results, actual recovery rollouts, and targeted horizon ablations.
Critical strengths
The method is intuitive and addresses a practically important problem. Allowing an initial correction before requiring agreement with demonstrated behavior is well motivated. Evaluation across transformer, flow-based, and diffusion-based policies supports the potential breadth of the approach. The trajectory and kinematic analyses provide useful context beyond success rates, and the manuscript appropriately acknowledges several limitations. Overall, the reported improvements make PARS a promising training strategy.
Critical weaknesses
-
Robustness of the reported improvements. Table 1 reports the best augmentation setting for each task, while Appendix D lists a single training seed. The separation between hyperparameter selection and final testing is not clearly described, and the comparability of tuning budgets remains unclear. This leaves uncertainty about the influence of selection and training randomness. Independent validation, multiple training seeds, and uncertainty estimates would strengthen the results. A shared augmentation configuration would also help assess reliance on task-specific tuning.
-
Evidence for actual recovery. Figure 2 evaluates proximity of predicted joint configurations to demonstrations. Comparing policies at the same off-nominal observations is useful, but this proxy does not establish successful recovery during execution. Contact dynamics, object state, and tracking errors can affect whether the motion restores task progress. Executing the policies from matched off-nominal simulator states and reporting subsequent success and recovery time would more directly support the central claim.
-
Missing ablations and limited scope. The experiments fix a 10-step free prefix within a 20-step prediction horizon and execute the full chunk before observing again. Varying prefix length and replanning frequency would clarify when suffix anchoring helps. Comparisons with alternative supervision schedules should use comparable losses and tuning budgets. The theory offers conditional local insight, while the experiments use low-dimensional simulated observations; broader claims about recovery and applicability should remain proportionate to this evidence.
Questions for authors
-
Hyperparameter selection: How were the task-specific augmentation settings selected? Were validation episodes separate from the final evaluation, and were comparable tuning budgets used for PARS and noise augmentation?
-
Robustness of the results: The manuscript reports a single training seed. Are additional results available to assess whether the improvements are consistent across seeds or less dependent on task-specific tuning?
-
Evidence of recovery: Beyond the predicted-trajectory analysis in Figure 2, is there evidence that PARS improves recovery during actual execution? For example, were the compared policies executed from the same off-nominal states to assess subsequent task success?