acceptodds
Under review as a conference paper at ICLR 2027

Beyond Validation Loss: Coverage and Reliability Dynamics in Robot Policy Pretraining

Abstract

In modern robot learning, generative robot policies are first pretrained through supervised learning from demonstrations, then refined through reinforcement learning, supervised finetuning, or other forms of posttraining. Yet conventional pretraining metrics, such as validation loss and average task success, provide an incomplete account of which policies support both effective direct deployment and subsequent posttraining. In this work, we study closed-loop pass@k, which evaluates the probability that k rollouts from the same initial condition attain at least one success. Thus, pass@1 measures mean task success; large-k pass@k measures whether success remains reachable under repeated sampling. Across a wide range of posttraining recipes, we show that large-k pass@k reveals posttraining differences not captured by pass@1. In particular, checkpoints with similar pass@1 but different large-k pass@k have different posttraining performance. Moreover, we show pass@k peaks at different checkpoints during supervised pretraining, revealing a coverage–reliability tradeoff over the course of training. Based on these observations, we propose two practical interventions: (1) a purely offline checkpoint-selection criterion that balances coverage and reliability without closed-loop evaluation, and (2) a generative data augmentation procedure that expands supervision beyond the finite demonstration set, preserving large-k pass@k while maintaining or improving pass@ and raising the observed coverage–reliability Pareto frontier. Together, our results motivate evaluation, checkpoint selection, and training procedures that explicitly account for posttraining.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, behavior cloning, policy finetuning, learning dynamics

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_NdYBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how supervised pretraining affects robot policies before and after finetuning. Using repeated rollouts from the same initial condition, it finds that longer training can improve pass@1 while reducing pass@k for larger k and hurting later finetuning. The authors propose a checkpoint selection rule based on validation action-error quantiles and an augmentation method that generates observation-action pairs. Experiments use diffusion policies on MimicGen and a VLA on RoboCasa, with posttraining settings including residual RL and supervised finetuning. RoboCasa uses an action-only augmentation variant.

Critical strengths
  • The comparison between checkpoints with similar pass@1 is useful. Their different finetuning outcomes show why average success alone can be misleading when choosing a pretrained policy. Seeing this in both MimicGen and RoboCasa makes the result more convincing.

  • The split-data experiments and frozen-encoder controls support the overfitting explanation. Replacing the late checkpoint’s low-noise denoising steps with earlier weights also recovers much of the lost coverage. These experiments help explain where the effect comes from.

Critical weaknesses
  • My main concern is the lack of experiments on real robots. All evaluation is in MimicGen and RoboCasa simulation. The paper motivates offline checkpoint selection partly by the cost of real robot rollouts, but does not test whether the selector or augmentation helps on hardware. It is also unclear whether the same coverage loss appears under real sensing noise and contact dynamics.

  • The RL results need a separate test of the learned policies: the reported curves use success over the latest 100 training episodes. For the selector, I would like a direct comparison with mean validation action MSE and low-noise denoising loss at matched finetuning budgets. Matching a similarly defined pass@k oracle does not show how much downstream performance the selection rule gains.

Questions for authors
  1. Are any real robot results available that show the same coverage–reliability tradeoff or improvements from the proposed selector and augmentation?
  2. When evaluated on shared held-out initial conditions, how does the selected checkpoint compare with checkpoints chosen by mean validation action MSE or low-noise denoising loss after the same finetuning budget?
4 comments
AuthorsOpenReview ↗

Real robot results

We thank the reviewer for the insightful comment and acknowledge the lack of real-robot experiments in the original submission. To test whether our simulation findings carry over to a real robot, we added experiments on a 6-DoF Push-T task using only a wrist-mounted camera (https://28544-rebuttal.github.io/#real-robot, Sec. 6 of the revised manuscript). We train a diffusion policy on 680 demonstration trajectories and repeat our offline analyses on 60 held-out trajectories. The best-of-k validation action MSE and error quantiles exhibit the same patterns observed in simulation, consistent with the coverage–reliability tradeoff.

We evaluate three checkpoints on the same 20 fixed initial configurations, with five rollouts per checkpoint per configuration (300 rollouts in total). Fig. 7(e) in the updated manuscript reports the following results.

Policy checkpointpass@1 ↑pass@5 ↑
Offline-selected baseline checkpoint (epoch 46)0.620.95
Final baseline checkpoint (epoch 200)0.520.75
Final JointAug checkpoint (epoch 200)0.631.00

The offline-selected baseline checkpoint outperforms the final baseline checkpoint by 10 percentage points on pass@1 and 20 percentage points on pass@5. JointAug further improves pass@5 to 1.00, with comparable pass@1 to the offline-selected baseline. Both checkpoints also exhibit more behavioral diversity and better recovery capability than the final baseline checkpoint, as illustrated in the videos on our website.

To examine whether checkpoint selection also benefits downstream world-model learning, we train two action-conditioned world models on the same demonstration data, augmented with 200 rollout trajectories from either the offline-selected or final baseline checkpoint. All rollouts are included regardless of success. We evaluate both models on the same 100 held-out trajectories collected from intermediate policy checkpoints. Table 3 reports the following results.

Rollout source for world-model trainingPSNR ↑, chunk 1LPIPS ↓, chunk 1PSNR ↑, chunk 2LPIPS ↓, chunk 2
Offline-selected baseline27.90.11225.20.136
Final baseline26.80.12024.10.145

Chunk 1 consists of four frames predicted from two ground-truth context frames, and chunk 2 consists of the next four frames predicted autoregressively. The world model trained with offline-selected checkpoint rollouts achieves higher PSNR and lower LPIPS in both chunks, suggesting that preserving policy coverage can also benefit downstream world-model learning.

AuthorsOpenReview ↗

RL train–eval gap and checkpoint selection by low-noise denoising loss

The RL results need a separate test of the learned policies: the reported curves use success over the latest 100 training episodes.

We thank the reviewer for raising this insightful question. We have added evaluation curves for held-out initial configurations to https://28544-rebuttal.github.io/#rl-gap and appendix C.2 of the updated manuscript. Every 50,000 environment steps, we evaluate the policy on the same 500 held-out initial configurations. When viewed side by side, the training and held-out evaluation curves show no noticeable performance gap.

For the selector, I would like a direct comparison with mean validation action MSE and low-noise denoising loss at matched finetuning budgets. Matching a similarly defined pass@k oracle does not show how much downstream performance the selection rule gains. When evaluated on shared held-out initial conditions, how does the selected checkpoint compare with checkpoints chosen by mean validation action MSE or low-noise denoising loss after the same finetuning budget?

We thank the reviewer for raising this insightful question! We have added comparison results and discussion to https://28544-rebuttal.github.io/#rl-gap and appendix C.2 of the updated manuscript***.***

Mean validation action MSE plateaus late in training, with its minimum within 0.3% of the final checkpoint's value, so selecting by it effectively gives the final checkpoint. We apologize for not clarifying this in the original submission. Please refer to Fig. 14(a), where best-of-k action MSE with (k=1) is exactly the mean validation action MSE. Selecting by this metric therefore gives the final-checkpoint baseline already included in our comparisons.

As for low-noise denoising score-matching loss (i.e., ε-prediction loss), first, it does not directly measure action MSE after the full sampling chain. Prior work likewise reports that validation denoising score matching loss or flow-matching loss is not significantly correlated with real-world task performance [1]. In contrast, our proposed coverage–reliability balance criterion uses sampled-action MSE, which correlates more strongly with task performance.

Furthermore, our Fig. 12 experiment only serves as a diagnostic experiment showing that the last two denoising calls account for most of the closed-loop pass@k tradeoff on one specific task, namely Threading D2. This does not establish that the same cutoff can be applied universally to other tasks. Moreover, selecting the noise-level cutoff is nontrivial. In Fig. 12(c), we distinguish the effects of different denoising bands through large-scale closed-loop evaluation. By contrast, our coverage–reliability balance criterion is fully offline and explicitly accounts for both favorable-tail sample quality (coverage) and typical and high-error samples (reliability) with no hand-tuned hyperparameters or thresholds.

We use the final two DDIM steps (same as Fig. 12(c)) as the cutoff for all five MimicGen tasks. For each task, we averaged the held-out ε-prediction MSE uniformly over all training diffusion timesteps at or below this cutoff and selected the checkpoint with the lowest average loss. This criterion selects epoch 100 for all five tasks. On Hammer Cleanup and Mug Cleanup, this happens to be the checkpoint with the highest pass@10, but on the remaining three tasks, epoch 100 is too early for either pass@1 or large-k pass@k to have reached its peak. Therefore, a fixed cutoff that works on one task might not identify a good checkpoint on the others.

Thank you again for your time and thoughtful feedback! Please let us know if further clarification or additional experimental results could help improve the score. We truly appreciate your input!

Reference:

  1. Allshire, Arthur, et al. "Scalable Behavior Cloning with Open Data, Training, and Evaluation." arXiv preprint arXiv:2606.27375 (2026).

Thank you for the additional experiments. The held-out RL evaluations address my concern about relying on training success, and the new comparison clarifies how the selector performs against the low-noise denoising baseline. The Push-T results also show that checkpoint selection and JointAug can improve rollout performance on hardware. Based on these additions, I am raising my rating to Weak acceptance.

AuthorsOpenReview ↗

Thank You for Your Feedback!

Thank you so much for taking the time to read our rebuttal and for raising your score. Your insightful feedback has really helped strengthen our submission!

Reviewer_r6TGOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper shows that over training, the generative robot policy tends to perform better in pass@1 while performs worse in pass@k on evaluation data. Consequently the paper coins the term a coverage-reliability tradeoff to describe this phenomenon. The paper first proposes an offline checkpoint selection criterion based on the generated-optimal action differences using a heldout set, specifically identifying the best through comparing the errors on the upper quartile and the lower quartile. The paper then proposes a data-augmentation technique that uses a generative model to generate observation-action pairs for training. Through empirical illustrations show that the selection criterion already performs better than simply choosing the last checkpoint, and the data augmentation generates data that enables pretraining and post-training to have higher coverage and reliability.

Critical strengths
  1. The pass@1 vs pass@k tradeoff is interesting and in fact relevant to the LLM breadth-validity tradeoff. The paper does a good job illustrating that this problem exists in training generative robot policies as well.
  2. The problem of choosing a good checkpoint for robotics control is a difficult problem [1, 2, 3] and the paper proposes a reasonable metric to choose in hindsight without rolling out each trained policy directly.

References

  • [1]: A. Mandlekar, et al., “What Matters in Learning from Offline Human Demonstrations for Robot Manipulation,” in Conf. Robot Learning, Nov. 202
  • [2]: L. Hussenot, et al., “Hyperparameter Selection for Imitation Learning,” in Proc. 38th Int. Conf. Machine Learning (ICML’21), July 2021, pp. 4511–4522.
  • [3]: T. Ablett, B. Chan, and J. Kelly, “Learning From Guided Play: Improving Exploration for Adversarial Imitation Learning With Simple Auxiliary Tasks,” IEEE Robotics and Automation Letters, vol. 8, no. 3, pp. 1263–1270, Mar. 2023.
  • [4]: J. Kleinberg, and M. Sendhil. "Language generation in the limit." Advances in Neural Information Processing Systems 37 (2024): 66058-66079.
Critical weaknesses

My only concern is that there is no theoretical justification as to why the chosen criterion is sound. It would be great if there is any sort of guarantees through making certain assumptions.

Questions for authors
  1. How does the chosen checkpoint compare against the best (pass@1 and pass@k) checkpoints?
  2. It seems like the gap between pass@1 and pass@k would start at the same point at epoch 0 (or at small training steps depending on the epochs). How does the comparison look in that scenario?
5 comments
AuthorsOpenReview ↗

Lack of theoretical guarantee

We fully acknowledge the reviewer’s point that a theoretical justification for the selection criterion would strengthen the paper. Our primary goal in this submission is empirical. Through extensive experiments, we characterize overfitting during generative robot policy pretraining and study interventions that yield better starting points for posttraining. We motivate the offline selection criterion from the learning dynamics we observe and evaluate it across tasks and dataset sizes. We appreciate the reviewer’s suggestion to derive a guarantee under explicit assumptions and see this as an important and valuable direction for future work, alongside the broader theoretical question of how pretraining dynamics shape posttraining.

AuthorsOpenReview ↗

How does the chosen checkpoint compare against the best pass@1 and pass@k checkpoints and early pass@1 and pass@k gap?

We thank the reviewer for the insightful question. We have added the comparison to https://28544-rebuttal.github.io/#selected-vs-best and appendix C.1 of the updated manuscript. Across the five MimicGen tasks, RL finetuning from our selected checkpoint yielded performance that was better or no substantially worse than that obtained by finetuning from either the checkpoint with the highest pass@1 or pass@10. However, using pass@1 or pass@10 for checkpoint selection in practice closed-loop evaluation of every candidate checkpoint. This can be prohibitively expensive on a real robot. Our coverage–reliability balance criterion selects a checkpoint entirely offline using held-out data, making it more practical for real-robot use, as we demonstrate in the next thread.

We appreciate the reviewer’s question on early pass@1 and pass@10 gap. The original MimicGen curves begin at epoch 50, with evaluations every 50 epochs. The RoboCasa curves begin at training step 10,000, with evaluations every 10,000 steps. We now also report results for MimicGen checkpoints before epoch 50 (https://28544-rebuttal.github.io/#early-gap). Across all five MimicGen tasks, pass@1 and pass@10 are both near zero at the earliest evaluated checkpoints, and pass@10 rises more sharply as training progresses. We have clarified the evaluation schedule in appendix sec H.1 in the revised paper.

AuthorsOpenReview ↗

Real robot results

To test whether our simulation findings carry over to a real robot, we added experiments on a 6-DoF Push-T task using only a wrist-mounted camera (https://28544-rebuttal.github.io/#real-robot, Sec. 6 of the revised manuscript). We train a diffusion policy on 680 demonstration trajectories and repeat our offline analyses on 60 held-out trajectories. The best-of-k validation action MSE and error quantiles exhibit the same patterns observed in simulation, consistent with the coverage–reliability tradeoff.

We evaluate three checkpoints on the same 20 fixed initial configurations, with five rollouts per checkpoint per configuration (300 rollouts in total). Fig. 7(e) in the updated manuscript reports the following results.

Policy checkpointpass@1 ↑pass@5 ↑
Offline-selected baseline checkpoint (epoch 46)0.620.95
Final baseline checkpoint (epoch 200)0.520.75
Final JointAug checkpoint (epoch 200)0.631.00

The offline-selected baseline checkpoint outperforms the final baseline checkpoint by 10 percentage points on pass@1 and 20 percentage points on pass@5. JointAug further improves pass@5 to 1.00, with comparable pass@1 to the offline-selected baseline. Both checkpoints also exhibit more behavioral diversity and better recovery capability than the final baseline checkpoint, as illustrated in the videos on our website.

To examine whether checkpoint selection also benefits downstream world-model learning, we train two action-conditioned world models on the same demonstration data, augmented with 200 rollout trajectories from either the offline-selected or final baseline checkpoint. All rollouts are included regardless of success. We evaluate both models on the same 100 held-out trajectories collected from intermediate policy checkpoints. Table 3 reports the following results.

Rollout source for world-model trainingPSNR ↑, chunk 1LPIPS ↓, chunk 1PSNR ↑, chunk 2LPIPS ↓, chunk 2
Offline-selected baseline27.90.11225.20.136
Final baseline26.80.12024.10.145

Chunk 1 consists of four frames predicted from two ground-truth context frames, and chunk 2 consists of the next four frames predicted autoregressively. The world model trained with offline-selected checkpoint rollouts achieves higher PSNR and lower LPIPS in both chunks, suggesting that preserving policy coverage can also benefit downstream world-model learning.

Thank you again for your time and thoughtful feedback! Please let us know if further clarification or additional experimental results could help improve the score. We truly appreciate your input!

Reviewer_ZP5iOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper discusses how pass@k performance involves over training: Large-k pass@k (coverage) peaks early and then declines while pass@1 (reliability) keeps improving or saturates. Ptertrained models with the same pass@1 performance can produce very different post-training dynamics (because they have different large-k pass@k).The authors attribute this observation to finite data overfitting and propose an offline checkpoint selection method that balances coverage and reliability. The paper also introduces a data augmentation method to expand the dataset to preserve large-k pass@k while maintaining or improving pass@1.

Critical strengths

This paper did a good job tracing the research story and empirically support every part of the story. for instance, Section 3 characterizes large-k pass@k and pass@1 differences during training and shows that different checkpoints empirically lead to different post-training curves. Additional experiments then diagnose why models with higher large-k pass@k enjoy stronger postraining performance despite similar pass@1. Section 4 then walks through how to choose a good model checkpoint for post-training. The paper was easy to follow (takeaways are clearly summarized at key points) and largely anticipated questions the reader might have throughout.

Critical weaknesses

The main weakness I see is that the augmentation method introduced isn't compared against any other augmentation approach (or even benchmarked against simply giving the agent more real data). While I understand that the augmentation method itself is not the main contribution, it is still a contribution and should be evaluated against other approaches. A naive baselines to consider is simply adding random noise to observations/actions. There are existing data augmentation approaches out there that generate higher-quality data too, like GTA [1[. If there is good reason to exclude such comparisons, please justify it in the paper.

  1. Lee et al. GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learning. Neurips 2024. https://arxiv.org/abs/2405.16907
Questions for authors

Why are no additional augmentation baselines considered in Section 5?

5 comments
AuthorsOpenReview ↗

Additional justification for JointAug

We thank the reviewer for suggesting broader augmentation comparisons! Our original motivation for choosing PostBC and TMRL was that both explicitly modify pretraining to preserve coverage or exploration for subsequent RL, directly matching our focus on how pretraining affects posttraining. We fully concede that comparisons with additional augmentation approaches would strengthen this part of the evaluation. Below, we clarify the motivation for JointAug and report additional experimental results. Appendix section F.5 and F.6 of the revised manuscript show the full results and discussion. A shorter summary with all the figures is at https://28544-rebuttal.github.io/#aug-baselines.

JointAug builds on a simple generative augmentation idea that has been explored in different forms by prior work such as [1,2]. Rather than selling it as a method, we want to emphasize how a joint generator can capture population-level structure shared across demonstrations. We use the action nearest-neighbor distance ratio to select generator checkpoints before overfitting. Sampling from these checkpoints gives the policy fresh, compatible observation–action pairs that reflect this shared structure and fill gaps between empirical training samples. The linked 2D illustration at https://28544-rebuttal.github.io/#jointaug-why explains this intuition.

AuthorsOpenReview ↗

Additional augmentation baseline results and discussion

GTA offers a valuable approach to generating high-return trajectories. High return and broad coverage are nevertheless distinct properties: high-return generation can potentially concentrate samples in a narrow set of conditions and reduce coverage. Our paper emphasizes preserving behavioral coverage as pretraining proceeds. We view GTA [2], GenAug/ROSIE [3,4], and CCIL [5] as valuable methods with complementary benefits: GTA directs generation toward higher returns, GenAug/ROSIE broaden visual variation, and CCIL supplies corrective supervision. Each of these mechanisms could further improve performance when combined with JointAug, which focuses on capturing population-level structure of expert behaviors. We have expanded our discussion of these connections and added the relevant citations in related work section in the updated manuscript.

GTA was developed for offline RL and uses return-conditioned trajectory generation. Our behavior-cloning setting uses only successful demonstrations, each with a sparse reward of 1, and does not use return conditioning. A comparison would therefore require adapting GTA’s return-guidance mechanism. Without return conditioning, GTA would share JointAug’s core idea of jointly generating observations and actions for policy training and could offer similar performance gains. We therefore discuss GTA as related work and additionally benchmark against CCIL and GenAug.

We evaluated CCIL with both state and image observations (https://28544-rebuttal.github.io/#ccil, appendix F.5 in updated manuscript). With state observations, CCIL largely improved the baseline Diffusion Policy’s performance. For the image-based experiments, we froze the visual encoder from the epoch-150 baseline checkpoint and applied CCIL to its low-dimensional embeddings. CCIL is difficult to scale to high-dimensional observation inputs because it requires an accurate local dynamics model and must analytically solve for a nearby starting state from which a perturbed action would reach the demonstrated next state. Even when applied in this lower-dimensional latent space, CCIL underperformed JointAug, which generates observation–action pairs directly.

We also compared against GenAug (https://28544-rebuttal.github.io/#genaug, appendix F.6 in updated manuscript). Our implementation uses Stable Diffusion to inpaint the table and background in each demonstration frame, varying surface appearance or adding distractors while preserving all robot and task-object pixels. During training, we replace the original frame with its edited version with probability 0.5 and pair it with the clean demonstrated action. GenAug’s background and distractor edits generate visual variation while preserving the robot pose, object poses, and demonstrated action. Across generated views, each robot–object configuration therefore remains paired with exactly the same action. We hypothesize with GenAug, the policy learns to ignore the randomized appearance and increasingly overfits to correlations between control-relevant cues and actions that are specific to the finite demonstration set. For instance, the policy may be overfitting to the mapping from robot pose to action observed in the finite training set. This could explain the lower final pass@1 and the steeper late-training decline in pass@10 on several tasks.

AuthorsOpenReview ↗

Real robot results

To test whether our simulation findings carry over to a real robot, we added experiments on a 6-DoF Push-T task using only a wrist-mounted camera (https://28544-rebuttal.github.io/#real-robot, Sec. 6 of the revised manuscript). We train a diffusion policy on 680 demonstration trajectories and repeat our offline analyses on 60 held-out trajectories. The best-of-k validation action MSE and error quantiles exhibit the same patterns observed in simulation, consistent with the coverage–reliability tradeoff.

We evaluate three checkpoints on the same 20 fixed initial configurations, with five rollouts per checkpoint per configuration (300 rollouts in total). Fig. 7(e) in the updated manuscript reports the following results:

Policy checkpointpass@1 ↑pass@5 ↑
Offline-selected baseline checkpoint (epoch 46)0.620.95
Final baseline checkpoint (epoch 200)0.520.75
Final JointAug checkpoint (epoch 200)0.631.00

The offline-selected baseline checkpoint outperforms the final baseline checkpoint by 10 percentage points on pass@1 and 20 percentage points on pass@5. JointAug further improves pass@5 to 1.00, with comparable pass@1 to the offline-selected baseline. Both checkpoints also exhibit more behavioral diversity and better recovery capability than the final baseline checkpoint, as illustrated in the videos on our website.

To examine whether checkpoint selection also benefits downstream world-model learning, we train two action-conditioned world models on the same demonstration data, augmented with 200 rollout trajectories from either the offline-selected or final baseline checkpoint. All rollouts are included regardless of success. We evaluate both models on the same 100 held-out trajectories collected from intermediate policy checkpoints. Table 3 reports the following results:

Rollout source for world-model trainingPSNR ↑, chunk 1LPIPS ↓, chunk 1PSNR ↑, chunk 2LPIPS ↓, chunk 2
Offline-selected baseline27.90.11225.20.136
Final baseline26.80.12024.10.145

Chunk 1 consists of four frames predicted from two ground-truth context frames, and chunk 2 consists of the next four frames predicted autoregressively. The world model trained with offline-selected checkpoint rollouts achieves higher PSNR and lower LPIPS in both chunks, suggesting that preserving policy coverage can also benefit downstream world-model learning.

Thank you again for your time and thoughtful feedback! Please let us know if further clarification or additional experimental results could help improve the score. We truly appreciate your input!

References:

  1. Wang, Ren, et al. "Prioritized generative replay." International Conference on Learning Representations. Vol. 2025. 2025.
  2. Lee, Jaewoo, et al. "Gta: Generative trajectory augmentation with guidance for offline reinforcement learning." Advances in neural information processing systems 37 (2024): 56766-56801.
  3. Chen, Zoey, et al. "Genaug: Retargeting behaviors to unseen situations via generative augmentation." arXiv preprint arXiv:2302.06671 (2023).
  4. Yu, Tianhe, et al. "Scaling robot learning with semantically imagined experience." arXiv preprint arXiv:2302.11550 (2023).
  5. Ke, Liyiming, et al. "Ccil: Continuity-based data augmentation for corrective imitation learning." International Conference on Learning Representations. Vol. 2024. 2024.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank You! A Summary of Our Rebuttal and Revisions

We thank all three reviewers for their time and constructive feedback! We have revised the manuscript to address each reviewer’s concerns with additional clarifications and new experimental results. A more concise text summary with figures and videos is available on our anonymous website https://28544-rebuttal.github.io/. We summarize the rebuttal response as below:

  1. Additional augmentation baselines (Reviewer ZP5i). We expanded the related work discussion and report the new baseline comparisons and discussion in Appendix Sections F.5 and F.6. A concise summary with visuals are at https://28544-rebuttal.github.io/#aug-baselines
  2. Comparison with the best pass@1 and pass@k checkpoints (Reviewer r6TG). Appendix Section C.1 compares RL finetuning from our offline-selected checkpoint with finetuning from the checkpoints with the highest pretraining pass@1 and pass@10. A concise summary with visuals are at https://28544-rebuttal.github.io/#selected-vs-best
  3. How the gap between pass@1 and pass@k emerges early in training (Reviewer r6TG). We added evaluations before epoch 50 on all five MimicGen tasks. A concise summary with visuals are at https://28544-rebuttal.github.io/#early-gap
  4. RL train–eval gap and checkpoint selection by low-noise denoising loss (Reviewer NdYB). Appendix Section C.2 compares training and held-out evaluation curves during RL finetuning and reports finetuning results and discussion for checkpoints selected by low-noise denoising loss. A concise summary with visuals are at https://28544-rebuttal.github.io/#rl-gap
  5. Real-robot experiments (Reviewer NdYB). Section 6 of the main text and Appendix Section H.8 report our results on real-robot Push-T with 6 DoF action. A concise summary with videos are at https://28544-rebuttal.github.io/#real-robot

Thank you again for helping us improve the paper! Please let us know if any further clarification could help improve the score. We would be grateful for the opportunity to address any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.