- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies on how to construct a dataset for behavioural cloning such that it covers a large variety of states that can be found at deployment time. This makes sure that the learned policies result more robust, for example to sim-to-real gaps.
The crucial idea is to perturb the environment while the learning process is being carried out. That is while the robot is trying to solve a manipulation task one of the objects can be rotated or shifted by the adversary. When the agent manages to solve the task despite the perturbations this trajectory is added to a dataset which is then used for behavioural cloning.
Experiments carried out in a simulated task (64 grocery packing tasks) and 3 bimanual real world tasks.
For additional efficiency, the authors choose to (D1) consider a success the accomplishment of each subtask under adversarial disturbances rather than considering as success only the solution of the full task. A second decision (D2) is to consider some degree of non-Markovianity in which the learner policy is not conditioned only on the current state but on a short portion of the history (i.e. 2 seconds window of recent history)
The experiments show that -DATA achieve higher success rate compared to the expert policy on both simulations and real robot tasks (see Table 1) and the performance keeps increasing with larger amount of training data and test-time compute (see Figure 5) . D2, i.e. the decision to add non Markovianity, helps especially in Long OOD tasks but less than considering subtasks as individual successes which seems to be the crucial ingredient for OOD generalization.
Finally, section 7 shows that -DATA also offers a better initial checkpoint for online finetuning compared to standard imitation learning.
Critical strengths
-
The method is simple and well explained.
-
I really enjoyed the design of the experiments which really aim at understanding which are the desirable characteristics of -DATA.
Critical weaknesses
-
why in Table 1 the success of Both the expert and -DATA is better Out Of Distribution compared to In Distribution? I would have expected the opposite
-
Concerning the tuna can example in Figure 4, can you clarify if -DATA ever saw an expert demonstration in which the tuna can was grasped successfully. I find it surprising if -DATA learned to do something that the expert systematically fails to do. Please elaborate on this point.
-
No baselines have been used for comparison. Why only standard BC is used as comparison and not recent variants that improves the coverage of BC such as PostBC ([1]).
-
I think that the need for introducing disturbances during the data collection time can make it way more complex than the one for standard behavioural cloning. The authors should be transparent about this limitation.
-
I think that there is an important missing citation ([2]) concerning the role of action chuncking and noise injection at data collection time.
I think that one weakness on which the authors should work on is that some plots are difficult to read, see for example my comment on Figure 6.
[1] Posterior behavioral cloning: Pretraining bc policies for efficient rl finetuning Wagenmaker et al. 2025
[2] Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Control, Zhang et al. 2025
Questions for authors
Q1) Does standard in Figure 5 refers to standard behavioural cloning?
Q2) I do not understand the legend of Figure 6a, it seems that the gray boxes in the legend (No history and 120 step history) are not plotted. If I understand correctly you mean that darker orange or darker blue means with history but this is just a guess and I think that the legend should do a better job at making this clear.
Q3) In Table 2, why does success decreases for expert data and progress decrease for both approaches? Moreover what is the meaning of the numbers in brackets (e.g. [48.1, 85.5]).
5 comments
Response to Reviewer HWDL (2/3)
"Why only standard BC is used as comparison and not recent variants that improves the coverage of BC such as PostBC" (W3)
Thanks for the great suggestion. We now have run PostBC, and Section 6 now reports the comparison. We midtrain a PostBC policy on the same standard demonstrations, starting from the same pretrained checkpoint. The policy uses an ensemble of 20 members trained on frozen policy features and a noise scale of α = 1.
| Zero-shot success | PostBC | Standard demonstrations | -DATA |
|---|---|---|---|
| Twelve OOD tasks | 64.0% | 62.7% | 79.2% |
| Eight ID tasks | 43.0% | 42.8% | 55.5% |
PostBC performs comparably to midtraining on standard demonstrations, and it underperforms -DATA on both splits. Upon visually inspecting rollouts, we do actually see that PostBC broadens the action distribution, but it does not add the transitions between abstract states that -DATA contains to improve recovery and correction behaviors upon deployment.
"the need for introducing disturbances during the data collection time can make it way more complex than the one for standard behavioural cloning" (W4)
Real-world collection of -DATA does in fact need a second person: two humans jointly collect each rollout, an operator and an adversary (Section 5). Each hour of real-world -DATA therefore takes two person hours.
To address this concern is precisely why we ran the data scaling experiments in Figure 5. This experiment showcases that training on half as much data can result in -DATA matching standard demonstrations: 2.6 hours of -DATA reach 60.3% success comparable to 58.7% with 5.2 hours of standard demonstrations (Section 6, Result 2). This means that at this budget, for the same total human time, we are able to get a matching policy. But in addition, we also find that continuing to scale -DATA further keeps on providing improvements, and -DATA stays ahead of standard demonstrations at every data budget from 1.3 hours onward (Figure 5). -DATA also provides for a better initialization and more effective online finetuning as shown by the new scaling law results in our response to Reviewer 8Dfn (Appendix L, Figure 19), which further justifies why -DATA is preferable despite needing two humans for collecting one datapoint. It is more data efficient, results in better initializations for online policy performance, and leads to improvements in zero shot performance.
"there is an important missing citation ([2]) concerning the role of action chunking and noise injection at data collection time" (W5)
Thank you for these references. The revised paper now cites Zhang et al. (2026) in Section 1 and in Section 4. Section 4 now states that injecting noise into the actions of the expert during data collection, or predicting action chunks, avoids the exponential compounding of imitation errors. Section 1 states that noise injection into the policy or the teleoperation interface broadens experience by perturbing actions, but offers limited control over semantically useful challenging states. Our protocol changes the environment or the task and records how the operator continues, and it does not perturb the actions of the operator. The result of Zhang et al. is a worst case bound, and we do not claim that errors from training on standard demonstrations compound exponentially. Our claim is that standard demonstrations lack the low-level behaviors needed to sustain long rollouts and to connect challenging states to states from which further progress is possible.
Our comparison with PostBC gives a similar picture. PostBC tries to make the action chunks at each state more robust for downstream adaptation, but it broadens the action distribution without improving connected behavioral coverage meaningfully (Section 4). These connecting behaviors matter for test-time scaling, zero-shot success and online improvement.
"some plots are difficult to read" and "I do not understand the legend of Figure 6a" (W6, Q2)
Thank you for pointing this out. We have fixed the legend of Figure 6(a) in the revised paper, and we hope it is clearer now.
Response to Reviewer HWDL (3/3)
"Does standard in Figure 5 refers to standard behavioural cloning?" (Q1)
Yes. "Standard" denotes the same pretrained policy midtrained on standard demonstrations, with the same objective, relabeling and training settings as the -DATA policy (Section 5, Appendix E).
"In Table 2, why does success decreases for expert data and progress decrease for both approaches? Moreover what is the meaning of the numbers in brackets (e.g. [48.1, 85.5])." (Q3)
The brackets are 95% intervals for the values after online finetuning. The interval for success is a Wilson interval, and the interval for task progress is a percentile bootstrap interval. The revised caption defines them now. For example, [48.1, 85.5] is the interval for the 70% success of the -DATA policy.
| Real-world packing (Table 2) | Before online finetuning | After online finetuning |
|---|---|---|
| -DATA, full-task success | 35% | 70%, interval [48.1, 85.5] |
| -DATA, task progress | 91.7% | 90.6%, interval [82.2, 97.2] |
| Standard demonstrations, full-task success | 20% | 15%, interval [5.2, 36.0] |
| Standard demonstrations, task progress | 66% | 53.3%, interval [41.7, 65.6] |
The policy trained on standard demonstrations relies on spurious cues in the scene, such as packing an item that the instruction does not name. Its attention does not fall on the instructed item, while the -DATA policy attends to it (Appendix I, Figure 14). Online finetuning on its own successful rollouts reinforces this reliance, which is why both its success and its task progress doesn't improve. With -DATA, we observe that the policy acts on the instructed object more reliably and recovers from failed grasps more often (Appendix I). Its success rises from 35% to 70%, and its task progress stays within its interval (91.7% to 90.6%, interval [82.2, 97.2]). Real-robot videos of both policies before and after online finetuning are on our anonymous project website (https://explr-beta.pages.dev/).
If our revisions address your concerns, we would be grateful if you could consider raising your score.
References
- Huang et al. (2025). Adversarial data collection: Human-collaborative perturbations for efficient and robust robotic imitation learning. arXiv:2503.11646.
- Wagenmaker et al. (2025). Posterior behavioral cloning: Pretraining BC policies for efficient RL finetuning. arXiv:2512.16911.
- Zhang et al. (2026). Action chunking and data augmentation yield exponential improvements in behavior cloning for continuous spaces. ICLR 2026 (arXiv:2507.09061).
Response to Reviewer HWDL (1/3)
Thank you for your feedback. To address your concerns we add the requested comparison with PostBC, explain why OOD success exceeds ID success in Table 1, clarify the tuna can example in Figure 4, and provide an additional comparison from the same initial state. We also explain the bracket values in Table 2 and correct the legend in Figure 6(a). Our new results show that β-DATA achieves higher success rates than PostBC, whose performance remains close to that of BC.
"Why in Table 1 the success of both the expert and β-DATA is better Out Of Distribution compared to In Distribution?" (W1)
Thanks for the question, we indeed agree this is confusing. The two splits contain different tasks, and OOD tasks are not necessarily harder than the ID tasks in terms of raw success rates. They are just different configurations from what was used for training of the policy. The OOD split has three Short, four Medium and five Long tasks. The ID split has three Medium and five Long tasks and no Short tasks (Appendix A). We also evaluated both policies on four additional short ID tasks with two subtasks each, and both succeed in all 200 trials (Appendix A). As such, higher success rate on the OOD split is true for both the β-DATA and standard demonstration trained policies, and not an artifact of any of the methods.
That said, on the Medium horizon tasks the order that you expect holds for standard demonstrations, which indicates that OOD success rates being higher is not a systematic pattern. The ID Long tasks are also much longer, with up to ten subtasks against at most five for the OOD Long tasks (Appendix A, Table 3).
"Can you clarify if β-DATA ever saw an expert demonstration in which the tuna can was grasped successfully?" (W2)
Figure 4 illustrates that broader low-level behavioral coverage matters even when the subtask is already specified. Both policies receive the same subtask instruction and know what to do, and they differ in whether their low-level execution completes the subtask (Section 6). To answer your question, yes. Both datasets are collected by human operators, and both contain demonstrations that grasp and pack the tuna can.
The paper does not claim that the policy trained on standard demonstrations systematically fails on the tuna can. Figure 4 shows one selected rollout of each policy from the same state on OOD Task 9 (Appendix A, Table 3), as an example where the policy trained on standard demonstrations fails and the β-DATA policy succeeds.
This difference does not rest on one rollout. In the new Figure 20 in Appendix M, from the same initial state in simulation, the β-DATA policy places a sugar box, and when the box slides over the tray rim, it regrasps the box, sets it flat in the tray, and packs all five objects. The policy trained on standard demonstrations holds the sugar box beside the tray until 100 s. In Figure 21 in Appendix M, on the real robot, both policies are instructed to pack a jar that lies on its side. The policy trained on standard demonstrations repeats the same grasp below the lid, and the jar stays on the table. The policy trained on β-DATA lifts the jar, and when the jar falls, it grasps the jar across its body and packs it.