acceptodds
Under review as a conference paper at ICLR 2027

Per-Task Retraining Noise in Robot Imitation Learning

Abstract

Imitation learning trains robot policies from human demonstrations that a person collects one episode at a time by teleoperating the robot. Because gathering them is slow and expensive, curation methods try to make that effort go further by scoring each demonstration and retraining the policy on only the highest-scoring ones. Across the six curation papers we audit, reported success-rate improvements run from 5 to 32 percentage points in simulation. Yet retraining a policy on unchanged data also moves its success rate, and none of these improvements was measured against that retraining noise. We train 701 policies on controlled demonstration subsets, spanning diffusion policy and ACT from scratch and the vision-language- action models SmolVLA and π0.5 fine-tuned. Each is evaluated on PushT, two bi- manual ALOHA tasks and two ten-task LIBERO suites, every batch pre-registered. Retraining on identical data shifts one LIBERO task’s success rate by a standard deviation of 20.4 percentage points and the ten-task average by 4.23, comparable to the reported gains. This per-task noise recurs in Meta-World, where one re- train of a small state-based policy solves a drawer-opening task 200 times out of 200 and another solves it 0 times. No score we built for a demonstration predicts whether keeping it improves the policy, so the safeguard is to measure the noise itself. Part of it comes from limited evaluation episodes and is captured by evaluating one policy many times. The rest comes from training itself and appears only when training is repeated, so eight retrainings of one fixed demonstration set are enough to capture it. This measurement tells a lab whether a new policy is a real improvement or a lucky retrain. We propose that every curation gain be quoted beside the retraining noise of its own setting, and we release the run records and pre-registrations that make that comparison possible.

Primary area:
applications to robotics, autonomy, planning
Keywords:
imitation learning, data curation, retraining noise, evaluation methodology

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_2jCFOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the scientific rigor behind data curation methods, and questions whether their gains are attributed to algorithm novelty or retraining noise. It conducts extensive experiments on 6 different data curation tasks, and reports results on their own reimplementation of these various methods. Overall, it finds that there is a lot of noise in performance depending on the seed that is used.

Critical strengths

The paper does highlight an important aspect of robotics research, which is the variance in seed to seed training. It does extensive training, and releases their code.

Critical weaknesses

Overall, my biggest concern is that I feel like this paper is an agglomeration of multi-seed experiments and reproductions of existing methods, without much novelty of it's own. While statistical rigor is indeed an important aspect of research, the contributions listed by the authors do not assist in actually advancing such rigor. The suite that they release is not practically usable, and there is no method that is clearly set for for the community, such as what was done for RL works in (Rliable)[https://araffin.github.io/post/rliable/].

Additionally, I have some concerns to the validity of the findings as a whole. Specifically, the authors mention starting ln387 that all evaluations happen on their implementations. It is an odd choice to not run on any published curation code, and that attributions of the findings to be up to reimplementation noise rather than the method family is a fair, yet unaddressed, critique of the paper. Also, ln467-471 very clearly state a defect in the results -- this seems highly unpolished.

As a side note, the manuscript, writing and figures feel heavily AI generated, without much attempt to improve legibility nor appearance. For example Figure 1 is very challenging to parse, with vague pairings between the symbols in ln 68-69 and the actual figure, and overlapping caption text in the figure itself. The key message is easily lost. I would highly encourage authors to re-do these plots, as well as the accompanying captions. Another example: the contributions list is long and scattered -- it reads more like a list of findings, instead of actual contributions. The full paper reads like a technical progression of experiments conducted.

Questions for authors
  1. What is the concrete takeaway the authors hope readers will take from their paper? How do authors envision their findings furthering the community as a whole? Please take into account that a lot of the findings were based on reimplementations, and thus there is likely confounding noise underlying all findings.
  2. What is the plan for authors to address the limitations and defect stated in the paper? (Such as ln 467-471?)
Reviewer_Z4FbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper conducts a systematic study, aiming to understand per-task retraining noise in robot imitation learning settings. The authors train 701 policies, including Diffusion Policy, ACT, and VLAs on PushT, Aloha, and LIBERO tasks, aiming to measure the noise itself coming from limited evaluations and training procedures. These findings will provide insights for researchers and practitioners to better benchmark and present their methods and experimental results.

Critical strengths
  1. The paper works on robot imitation learning, which is a very important and timely research topic;

  2. The authors conduct comprehensive experiments to present their findings and solidify their conclusions.

Critical weaknesses
  1. The writing is extremely hard for me to follow and understand. Although the whole story is quite straightforward, the authors self-define a lot of strange concepts, such as "cells", "wave", "mask", "ladder", "licensed", and so on. I don't understand why presenting such a straightforward story needs this level of massive concept definitions. The whole paper is not very organized.

  2. The authors' contributions are not very focused. The authors list a lot contributions in the paper's introduction section, which makes the central claim of this paper very ambiguous. It is more like a statistical report instead of a coherent paper. Plus, I don't understand what kind of insights or recommendations the authors aim to convey the researchers or practitioners based on the paper's findings. It seems that the only recommendation is training with more seeds and evaluating more episodes, which is already widely known.

  3. I am very concerned whether the retraining noise is caused by insufficient training. The authors seem to mention in the appendix that they directly adopt the default training configs from LeRobot and find that when training stops, the loss is still decreasing. This is very worrying message. I believe the authors should illustrate how they select training configs for each method very carefully and how they make sure that the retraining noise is not caused by insufficient training.

Questions for authors

See the weaknesses I list.

Reviewer_9DW8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies retraining variability in robot imitation learning using 701 trained policies across several architectures and benchmarks. It shows that suite averages can hide substantial variation on individual tasks, examines demonstration scoring and subset selection, and recommends measuring setting-specific retraining noise when reporting curation gains.

Critical strengths
  • The repeated-training experiments provide useful evidence that task-level performance can vary substantially even when the training data are unchanged. This matters for evaluating improvements that suite averages may obscure.

  • The authors distinguish training variability from evaluation noise and explicitly report failed predictions and withdrawn conclusions. This transparency strengthens the empirical contribution.

Critical weaknesses

The sample-size recommendations are not adequately calibrated for small samples. Appendix H claims that two seeds per arm resolve a 10-point effect with 80% power using a known-variance normal approximation. With the stated standard deviation of 3.51, a conventional two-sided two-sample t-test has approximately 36% power. The testing procedure and uncertainty in the variance estimate need to be accounted for.

Questions for authors
  • What statistical test supports the two-seeds-per-arm recommendation, and how do the required sample sizes change when variance-estimation uncertainty is included?

  • What evidence connects the LIBERO non-additivity result to the PushT scoring failures? Can the conclusions be restricted to the implementations and settings actually tested?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.