acceptodds
Under review as a conference paper at ICLR 2027

Noise-Gated Learning from Prior Experience for Generative Control

Abstract

Learning generative policies via behavior cloning demands high-quality, task-specific demonstrations that can be time-consuming to collect. Prior experience, including policy rollouts and data collected on other tasks, is more readily available and contains useful information such as environment dynamics, but can harm task performance if naively imitated. We introduce Noise-Gated Diffuser (NGD), a generative control framework that selectively draws supervision from prior experience beyond limited expert demonstrations. NGD jointly models actions and a compact future-state latent with independent noise levels. These define a two-dimensional noise plane that connects behavior imitation and dynamics modeling through a continuum of learning objectives. We use noise thresholds to select which regions receive supervision from each data source, with a learned embedding making the source identity explicit to the model. Across simulated and real-robot manipulation tasks with limited expert demonstrations, NGD consistently benefits from prior experience, with intermediate noise thresholds yielding the strongest average performance. On the real-robot tasks, NGD achieves nearly four times the average success rate of expert-only Diffusion Policy. Moreover, with expert demonstrations held fixed, the performance of \ours increases linearly with the amount of prior experience, highlighting its promise for scaling generative control with diverse trajectory collections.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Generative Control, Robot Learning, Learning from Prior Experience and Suboptimal Data

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TfDNOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces Noise-Gated Diffuser (NgD), a method to train diffusion policies on a combination of expert and broader sub-optimal experience data. NgD predicts future states (in latent space) and actions conditioned on current sensor observations. It choose a noise-gate parameter for learning from broad experience, which restricts learning of the denoiser to high-noise timesteps of the diffusion process. Expert demonstrations are used normally in the diffusion process.

Critical strengths
  • Learning from Sub-Optimal Data is a Topical Problem: As robotics datasets scale, its important to study how to use the broad experience data which might be sub-optimal but still useful learning.

  • Noise Gating in Diffusion is an Intuitive Idea: Restricting sub-optimal data to learning only when they are in the high-noise timesteps of diffusion makes sense. This only affects the early denoising process which is more important for high-level guidance and semantics. The expert demonstrations can guide the policy.

Critical weaknesses
  • Positioning of Contribution Relative to Prior Work: The idea of utilizing sub-optimal data by noise-gating has been explored by prior works. X-Diffusion [1] restrict cross-embdoiment data mismatched in dynamics to high-noise timesteps. Ambient Diffusion Policy [2] restricts sub-optimal data between a low-noise and high-noise threshold. These works should be cited and compared against. It's not clear if the paper introduces new methods compared to these papers.

[1] Maximus A. Pace, Prithwish Dan, Chuanruo Ning, Atiksh Bhardwaj, Audrey Du, Edward W. Duan, Wei-Chiu Ma, Kushal Kedia. "X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations." ICRA 2026. arXiv:2511.04671.

[2] Adam Wei, Nicholas Pfaff, Thomas Cohn, Arif Kerem Dayı, Constantinos Daskalakis, Giannis Daras, Russ Tedrake. "Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics." arXiv:2606.12365, 2026.

  • Manual Tuning of Noise Gating Parameter: Compared to [1] and [2], this work has to manually tune the noise gating parameters. These other works instead learn a classifier which can decide which noise level for each data sample seperately depending on how close they are to expert data.

  • Limited Realworld Experiments: The method is validated on 2 quasi-static manipulation tasks. How does performance vary with more dynamic tasks?

  • Minimal Performance Improvements Relative to Data Pooling: The gains from noise-gating are minimal compared to just pooling both data sources and training on the mixture.

Questions for authors
  • Can you clarify the positioning of this work relative to prior noise-gating approaches for generative control? [1,2]
  • Does the noise gating parameter need to be manually specified?
  • Can the noise gating parameters be applied differently to different samples and domains?

[1] Maximus A. Pace, Prithwish Dan, Chuanruo Ning, Atiksh Bhardwaj, Audrey Du, Edward W. Duan, Wei-Chiu Ma, Kushal Kedia. "X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations." ICRA 2026. arXiv:2511.04671.

[2] Adam Wei, Nicholas Pfaff, Thomas Cohn, Arif Kerem Dayı, Constantinos Daskalakis, Giannis Daras, Russ Tedrake. "Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics." arXiv:2606.12365, 2026.

3 comments

We thank Reviewer TfDN for their constructive feedback. We kindly refer the reviewer to our general response for the conceptual differences from X-Diffusion and Ambient Diffusion Policy (item 1), the baseline comparison (item 2), and the extended real-robot experiments with a fixed threshold, including a new deformable task (item 3). We also address the listed concerns below:

  • On the positioning of NGD relative to prior work (W1, Q1): X-Diffusion [1] and Ambient Diffusion Policy (AmbientDP) [2] both optimize suboptimal samples at noise levels where they become indistinguishable from the noised expert ones. In contrast, NGD neither restricts denoiser training to high-noise diffusion timesteps nor uses distributional indistinguishability to admit prior supervision. Noise thresholds in NGD select conditional action-outcome prediction objectives by determining how much information in the conditioning modality is available for prior data to supervise the prediction of the other. For example, a high future-state threshold demands the future state to be nearly clean for prior data to supervise the prediction of the action chunk, regardless of the action’s own noise level. We discussed AmbientDP in our initial related work. We have added X-Diffusion and discussed how these prior works differ from our approach in the revised related work.

  • On the Ambient Diffusion Policy baseline (W1): We have updated Table 1 to include results from AmbientDP on simulated benchmarks following the same training and evaluation protocols as NGD. For fair comparisons, we swept noise thresholds per benchmark within a budget similar to NGD and report AmbientDP results using the best benchmark-level threshold. We observe that NGD effectively outperforms AmbientDP across the evaluation benchmarks. Further details are provided in Appendix B.

    CanSquareTransportTool HangCoffeeBox CleanupCanSorting
    AmbientDP73.8 ± 1.748.4 ± 0.869.8 ± 5.429.2 ± 1.550.7 ± 7.432.3 ± 1.073.8 ± 1.0
    NGD77.7 ± 1.456.7 ± 2.775.1 ± 2.542.4 ± 3.550.8 ± 3.443.5 ± 6.884.3 ± 1.3
  • On the manual threshold selection (W2, Q2): While X-Diffusion [1] and AmbientDP [2] train a task- or dataset-specific classifier to decide the threshold for the high-noise interval, AmbientDP manually selects the threshold for the low-noise interval in real-robot experiments, and reports the difficulty of using a classifier-based approach for identifying a meaningful low-noise threshold in Appendix H.1 of [2]. Although the noise threshold is also a hyperparameter to be selected in our framework, we demonstrate the robustness of our approach by using a fixed threshold across extended real-robot evaluation settings, where we use a new set of prior trajectories from DROID [3] and introduce additional evaluation configurations for the two initial tasks along with a new real-robot task. In the results below, we observe that a fixed threshold of 0.6 in our approach robustly transfers across different prior data and yields non-trivial improvements on various real-robot tasks and evaluation settings, reducing the need for hand-picking thresholds across task settings.

  • On an additional real-robot dynamic task (W3): We present additional results on a deformable object manipulation task (Towel Hanging) in our extended real-robot evaluation. The task requires a robot arm to grasp and successfully mount a towel onto a rack. Throughout the task, grasping locations and contact forces dynamically alter the towel’s shape, directly influencing the required execution of the hanging motion. NGD with a fixed threshold of 0.6 remains robust and effectively improves task performance, as shown by the results below. We also provide the evaluation videos for this task on an anonymous website: https://ngd-rebuttal.pages.dev/.

    Success RateJoint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
    CoffeePod & Gear (Extended)65.7%60.0%82.9%
    Drawer Cleanup (Extended)45.7%62.9%80.0%
    Towel Hanging48.0%48.0%76.0%

[1] Pace et al. X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations. ICRA, 2026

[2] Wei et al. Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics. CoRL, 2026

[3] DROID Team. DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset. RSS, 2024

  • On minimal gains relative to naive pooling (W4): We would like to clarify that naive pooling generally degrades performance on simulated benchmarks, as shown by “NP” columns in Table 1. Across three benchmarks, NGD significantly outperforms both DP and Joint State-Action Models trained by naively pooling both data sources. Compared with Joint State-Action Models trained on naively pooled data, NGD improves average performance by 42.8%, 18.1%, and 78.9% on Robomimic, DexMimicGen, and real-robot evaluations. In our extended real-robot experiments above, NGD also substantially outperforms both its ungated variant (NGD w/ τ=0\tau=0) and Joint State-Action Models naively trained on pooled data.

  • On the adaptive use of noise-gating parameters across samples and domains (Q3): NGD’s gating formulation allows thresholds to vary across samples or domains, and using a shared threshold is a design choice rather than a requirement. In our experiments, a fixed threshold of 0.6 already achieves strong real-robot performance across different evaluation settings and tasks, suggesting that per-sample adaptation is not necessary for effective use of prior data with our approach. Meanwhile, we agree that developing an adaptive threshold-selection method is a promising direction for future work and have included this discussion in the revised Appendix D.

AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer TfDN,

As the discussion period is closing soon, we wanted to send a gentle note to check whether our rebuttal and the revised manuscript have addressed your concerns. If anything remains unclear, we would be glad to provide further clarification. Otherwise, we would greatly appreciate it if you could let us know whether the updates resolve your questions as you finalize your assessment. Thank you again for your time and thoughtful review.

The Authors

Reviewer_xcA2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a generative policy that learns from limited expert demonstrations and additional prior experience (including suboptimal rollouts and demonstrations from other tasks). The method jointly models action chunks and compact future-state representations using flow matching with independent noise levels. A noise-gating mechanism selectively applies action and future-state prediction losses to prior data. This aims to extract useful dynamics information without imitating undesirable behaviors. The method also uses source conditioning to distinguish expert demonstrations from prior experience. Experiments on RoboMimic, DexMimicGen, and real-robot manipulation tasks show improvements over several baselines.

Critical strengths

NGD improves performance across seven simulated and two real-robot manipulation tasks without requiring quality labels or filtering of prior trajectories. The improvements are particularly important on challenging tasks such as Box Cleanup and Drawer Cleanup (where expert-only learning performs poorly).

The method integrates selective supervision from prior experience into a joint state-action generative model using noise thresholds and source conditioning. The paper studies threshold selection, prior-data scaling, and different sources of prior experience. This provides useful empirical insights into learning from heterogeneous trajectory datasets.

Critical weaknesses
  1. “Our key insight is that varying the corruption level of the conditioning input continuously interpolates between objectives for task-specifi behavior modeling and task-agnostic dynamics prediction.” I do not think paper supports this claim.

The action chunk has 10–24 steps, but the only outcome information is one compact latent of the final frame. Many action sequences lead to the same final frame. p(A | S, S′) on prior data mostly reflects which path the behavior policy took. Supervising actions this way is goal-conditioned imitation of prior actions (not task-agnostic inverse dynamics).

Appendix E: when prior data supervises only the future-state loss, the result (47.3%) is no better than the expert-only joint model (~46.5%). The gain comes from the action loss (which is the behavior-dependent part).

Figure 2 shows that intermediate thresholds (tau 0.6-0.7) outperform tau = 0.999. This concentrates supervision near clean-conditioned dynamics objectives. However, this result does not necessarily mean that intermediate thresholds improve the learning of transferable dynamics. It might be the case that they allow the model to exploit additional behavioral information from prior trajectories. The paper does not have experiments that distinguish these explanations

The paper explains the gains as “reusable action primitives” (Sec. 4.5) and “partially useful motions” (Sec. 4.3), which is behavior transfer. Higher success rates do not show that the claimed mechanism is what is working.

  1. The effect of gating mechanism is relatively small. In Table 1, NGD tau=0, and NGD tau_b scores are close to each other. In many cases, the gain is in within one std. The largest gain seems to coming from source conditioning, e.g., Table 1 joint state action model results. Similarly, real-robot numbers are 20 trials with one seed. The gating gain (from 75 to 85) is 2 trials out of 20, and tao was picked from 8 values on these same trials. That’s why, “NGD achieves nearly four times” is overselling the method.
Questions for authors

The paper attributes its gains to transferable dynamics (Sec. 3.2). Secs. 4.3/4.5 explain them through reusable motions, and Appendix E shows that State-Loss-only (47.3%) performs similarly to the expert-only joint model (~46.5%). Predicting a 10–24-step action chunk from a single future-state latent may primarily learn the behavior-dependent distribution p(A | S, S′). This resembles hindsight goal-conditioned imitation rather than task-agnostic dynamics learning. Can the authors compare against a source-conditioned diffusion policy trained with hindsight goals psi(o_{t+H}) and goal dropout, evaluated without access to ground-truth future observations? If this baseline matches NGD, it would weaken the proposed dynamics-transfer explanation and change my assessment.

Figure 2 shows that intermediate thresholds outperform thresholds near 1. What evidence supports the interpretation that partially corrupted conditioning improves transferable dynamics learning, rather than simply introducing more behavioral information from prior trajectories?

Table 1 shows relatively small improvements from NGD with tau=0 to NGD with tau=tau_b on several tasks (often comparable to the reported standard deviations). Can the authors provide stronger statistical evidence that noise gating consistently improves performance beyond joint state-action modeling with source conditioning?

In the real-robot experiments, gating improves success from 75% to 85%, corresponding to only two additional successes out of 20 trials. Since the best threshold was selected from eight candidates on the same evaluation benchmark, can the authors report results with additional trials, multiple training seeds, and a held-out evaluation set using a fixed threshold?

4 comments

We thank Reviewer xcA2 for their helpful comments and feedback. We kindly refer the reviewer to item 3 of our general response for the extended real-robot experiments, which compare NGD with and without noise gating under a fixed threshold on additional configurations and a new task. We seek to address the reviewer's concerns below:

  • On the “task-agnostic dynamics” (W1): We thank the reviewer for pointing this out! We agree that the chunk-level inverse dynamics modeled by NGD, as a form of multi-step inverse dynamics [1,3,4], is closely related to hindsight goal-conditioned imitation [2,3], and can depend on the behavior policy of the prior data. While our use of “task-agnostic” was intended to describe the reuse of local relations between action chunks and their outcome, rather than independence from the data-collection policy, we acknowledge the potential confusion and have revised "task-specific behavior modeling" to "direct behavior imitation" and "task-agnostic dynamics prediction" to "chunk-level dynamics prediction" in Figure 1 and Section 3.2. Direct behavior imitation refers to imitating the recorded behavior from the current observation alone, while chunk-level dynamics prediction refers to modeling the conditional relations between an action chunk and its outcome in both the forward and inverse directions. We also clarify in Section 3.2 how chunk-level inverse dynamics allows prior trajectories to supervise reusable motion and how it differs from direct behavior imitation.

  • On the interpretation of performance gains at intermediate thresholds (W1, Q2): As discussed above, chunk-level dynamics in NGD covers both the forward direction, which captures state transitions, and the inverse direction, which allows prior trajectories to supervise reusable motion. We therefore do not view chunk-level dynamics learning and behavior transfer as competing explanations, and do not attribute the gains at intermediate thresholds solely to forward dynamics learning. Intermediate thresholds require the model to extract action-outcome relations from incomplete information. In this regime, prior trajectories teach the model how to move toward a roughly specified outcome, providing reusable motion, as well as what outcome a partially noised action chunk leads to. Our interpretation is that the structure learned from both forward and inverse directions can inform the generation process before either the action chunk or the future state is fully resolved, and together lead to gains. We have provided additional clarifications on this in Section 3.2. Moreover, Appendix E shows that neither action supervision nor future-state supervision alone explains the NGD performance at intermediate thresholds. This also suggests that supervision from both forward and inverse directions has complementary effects on improving performance when jointly optimized.

  • On the suggested goal-conditioned baseline (W1, Q1): We implement the suggested hindsight goal-conditioned imitation baseline with source conditioning and a goal dropout rate of 0.5, using the same diffusion policy backbone. The results below are obtained using the same training data and evaluation protocols. We observe that the two approaches obtain competitive performance on Tool Hang, showing that reusable motion transferred from prior behavior is helpful. However, our approach surpasses the baseline on the remaining tasks and achieves a non-trivial improvement on Box Cleanup and Can Sorting. This indicates that action supervision from prior data, and the reusable motion it provides, does not fully account for the gains of NGD. Consistent with Appendix E, this suggests action and future-state supervision have complementary effects when combined. We have included this comparison in Appendix F.

    CanSquareTool HangBox CleanupCan Sorting
    Hindsight Goal-Conditioned Imitation75.3 ± 1.052.5 ± 1.143.9 ± 0.936.7 ± 6.376.8 ± 3.6
    NGD77.7 ± 1.456.7 ± 2.742.4 ± 3.543.5 ± 6.884.3 ± 1.3

[1] Lamb et al. Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models. TMLR, 2023

[2] Andrychowicz et al. Hindsight Experience Replay. NeurIPS, 2017

[3] Paster et al. Planning from Pixels using Inverse Dynamics Models. ICLR, 2021

[4] Tian et al. Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation. ICLR, 2025

  • On the additional real-robot evaluation (Q4): To strengthen real-robot experiments, we report results on another 15 initial object configurations for both CoffeePod & Gear and Drawer Cleanup. Additionally, we provide results on a new real-robot task (Towel Hanging) focusing on deformable object manipulation. To make our setting more practical and held out from our initial threshold selection, we use 200 demonstrations from DROID [5], an existing robot manipulation dataset, as our prior trajectories. We use a fixed noise threshold of 0.6 throughout all experiments. The results below show that NGD with τ=0.6\tau=0.6 consistently achieves the best performance across these held-out evaluation settings without threshold retuning, demonstrating the practical effectiveness of a fixed threshold for our approach. We also provide real-robot evaluation videos on an anonymous website: https://ngd-rebuttal.pages.dev/.

  • On the stronger evidence of the gains from noise gating (W2, Q3): As shown in the results below, we perform extended real-robot evaluations with externally collected prior experience and a fixed noise threshold, in which we also provide additional evaluation trials and introduce a new deformable manipulation task (Towel Hanging). The results show that noise gating consistently provides gains that are greater than the improvements from source-conditioning. Specifically, noise gating achieves 38.2%, 27.2%, and 58.3% relative improvements on CoffeePod & Gear, Drawer Cleanup, and Towel Hanging, respectively, while source-conditioning alone no longer provides stable improvements across tasks.

    CoffeePod & GearJoint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
    Original Evaluation Set65.0%65.0%85.0%
    Additional Evaluation Set66.7%53.3%80.0%
    Overall65.7%60.0%82.9%
    Drawer CleanupJoint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
    Original Evaluation Set50.0%65.0%80.0%
    Additional Evaluation Set40.0%60.0%80.0%
    Overall45.7%62.9%80.0%
    Towel HangingJoint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
    Success Rate48.0%48.0%76.0%
  • On the overclaim (W2): We have revised the claim in the abstract and introduction to focus on the gains achieved by noise gating with a fixed threshold in our extended real-robot evaluations.

[5] DROID Dataset Team. DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset. RSS, 2024

Thank you for the response and the additional experiments.

The terminology change addresses my main concern in W1. "Chunk-level dynamics prediction" is more accurate, and I appreciate the acknowledgment that it is related to hindsight goal-conditioned imitation. The DROID experiments also provide much stronger evidence that gating helps.

I still have a few questions. My question about the contribution of gating concerned simulation, but the response focuses on real-robot results. The simulation results in Table 1 still use the best threshold from the sweep in Figure 2. Could you report the results with a fixed τ = 0.6 across all simulated tasks, alongside the ungated model? DexMimicGen already uses 0.6, so this only changes the Robomimic numbers. Using the existing runs would be useful; additional seeds would strengthen the comparison if feasible.

Thank you for running the hindsight baseline. Why are Transport and Coffee missing? Also, was goal dropout applied to prior samples, or only to expert samples? Prior samples with dropped goals receive ordinary source-conditioned imitation supervision, so this detail matters. The baseline is fairly close to NGD on several tasks, and I would like to understand the comparison better.

Could you also add the expert-only joint model for Towel Hanging and the additional evaluation configurations? This would show whether gated DROID data improves over expert-only learning in these settings, rather than recovering performance lost through pooling or ungated training.

I also do not see results from multiple training seeds on the real robot. If these are not feasible, please acknowledge this as a limitation.

Finally, the threshold was selected using the original real-robot configurations. Only the additional configurations and Towel Hanging are held out with respect to threshold selection. Please make this distinction clear in Section 4.5.

The simulation results with a fixed threshold, the missing baseline tasks, and the expert-only reference matter most to me. I am willing to raise my score if they are addressed.

AuthorsOpenReview ↗

Response to Reviewer xcA2

We thank the reviewer for the follow-up questions and are glad that our initial response has helped address their concerns. We provide additional results and clarifications below:

Robomimic results with τ=0.6\tau=0.6

We have updated Table 1 to report the Robomimic results achieved at τ=0.6\tau=0.6. We also present the results below, for the reviewer’s convenience. Although the average performance at τ=0.6\tau=0.6 is slightly lower than at τ=0.7\tau=0.7, both settings perform competitively across individual tasks. τ=0.6\tau=0.6 achieves the highest average performance in the updated Table 1.

Success RateCanSquareTransportTool Hang
NGD (τ=0\tau=0)75.9 ± 1.156.5 ± 1.270.6 ± 2.339.7 ± 3.6
NGD (τ=0.6\tau=0.6)77.3 ± 1.355.6 ± 1.675.7 ± 0.242.0 ± 1.3
NGD (τ=0.7\tau=0.7)77.4 ± 1.456.7 ± 2.775.1 ± 2.542.4 ± 3.5

Hindsight goal-conditioned imitation baseline

During the baseline training, we apply goal dropout on both expert and prior samples. Thus, prior samples can receive source-conditioned supervision with their goals dropped out. We provide the baseline comparisons on Transport and Coffee, and report the full results compared to NGD (τ=0.6\tau=0.6) below. The baseline performs strongly on both Transport and Tool Hang, suggesting the importance of behavior transfer in these tasks. Compared to the baseline, NGD (τ=0.6\tau=0.6) learns both forward and inverse chunk-level dynamics at intermediate noise levels. It achieves higher success rates averaged over all simulated tasks, with greater improvements on Box Cleanup and Can Sorting. This demonstrates the effectiveness of learning both directions of action-outcome relations and the robustness of our approach under a fixed noise threshold. We have also updated the baseline comparison in the revised Appendix F.

CanSquareTransportTool HangBox CleanupCoffeeCan Sorting
Hindsight Goal-Conditioned Imitation75.3 ± 1.052.5 ± 1.176.7 ± 4.543.9 ± 0.936.7 ± 6.348.7 ± 4.276.8 ± 3.6
NGD (τ=0.6\tau=0.6)77.3 ± 1.355.6 ± 1.675.7 ± 0.242.0 ± 1.343.5 ± 6.850.8 ± 3.484.3 ± 1.3

Performance of the expert-only joint model on extended real-robot experiments

Below we report expert-only joint model performance for all evaluation settings. We find that naively pooling with DROID demonstrations yields a modest improvement over expert-only training. NGD enables further improvement through noise gating with τ=0.6\tau=0.6 and effectively outperforms the baselines, including the expert-only joint models. We have also updated Figure 5 to include the expert-only joint model baselines.

CoffeePod & GearJoint State-Action Model (Expert-Only)Joint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
Original Evaluation Set55.0%65.0%65.0%85.0%
Additional Evaluation Set40.0%66.7%53.3%80.0%
Overall48.6%65.7%60.0%82.9%
Drawer CleanupJoint State-Action Model (Expert-Only)Joint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
Original Evaluation Set25.0%50.0%65.0%80.0%
Additional Evaluation Set33.3%40.0%60.0%80.0%
Overall28.6%45.7%62.9%80.0%
Towel HangingJoint State-Action Model (Expert-Only)Joint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
Success Rate over 25 trials40.0%48.0%48.0%76.0%

Multiple training seeds for real robot experiments

Due to the cost of real-robot training and evaluation, we report real-robot results over multiple evaluation trials from a single training seed per method. We have updated the Table 1 caption to indicate that the real-robot results come from one training seed and have acknowledged this limitation in Appendix D.

Held-out clarification in Section 4.5

We have updated Section 4.5 to clarify that the additional evaluation set and Towel Hanging are held-out settings for τ=0.6\tau=0.6.

We hope these additions help address the reviewer's remaining concerns, and we would be happy to provide further clarification if any questions remain. Thank you for your time and constructive feedback!

Reviewer_5aRhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes NGD, a method that uses suboptimal rollouts and demonstrations from other tasks to improve policy learning with limited expert data. The method jointly predicts action chunks and a compact representation of the future state, with independent noise levels for both. These noise levels are then used to control which losses are supervised by prior data, while expert demonstrations supervise all noise levels. A source embedding indicates whether the data comes from expert demonstrations or prior experience. The authors show improvements in success rate on seven simulated tasks and two real-world manipulation tasks. They also study the effect of the noise thresholds, the amount of prior data, and the contribution of action and future-state supervision.

Critical strengths
  • NGD uses expert demonstrantions and experience from prior trajectories to jointly model actions and a future state latent. NGD controls the amount of supervision that comes from prior experience in order to prevent the policy from imitating suboptimal actions. Prior data doesn't need success labels, which allows the use of unsuccessful rollouts with a limited set of expert demonstrations.
  • The experiments show improvements in simulation and real-world robot evaluation. The authors also compare with the same joint model and source conditioning but without gating (τ=0\tau=0), which helps understand how much the gating contributes to the improvements. Ablations in Appendix E also show that using both action and future-state supervision works better than using only one of the losses.
Critical weaknesses
  • The authors mention Ambient Diffusion Policy (ADP) and describe it as “closest to our setting”, but never compare to it. The authors seem to discard it because it is an action-only diffusion model, but the setting addressed is very similar, ie. learn from limited expert data and suboptimal experience. They also use a noise-dependent supervision.

  • The choice of the noise threshold seems important for the reported improvements. Table 1 uses the best threshold from Figure 2, but it is not clear if separate evaluations were used to choose the threshold and report the final results.

Questions for authors
  • Could you explain why you didn't compare with Ambient Diffusion Policy? What do you think is the performance using the same expert and prior data, a similar backbone, and similar budgets for training and choosing the threshold?
  • Could you explain how you choose the thresholds? Are the configurations or evaluation runs used for Table 1 separate from those used in the threshold sweep?
  • How are the noise levels sampled inside the gated regions and how are the losses normalized? Does changing τ\tau also change how much weight the prior data gets?
  • Could you explain how you choose the 48 initial environement configurations for collecting prior data? Are they shared between all the baselines ? Appendix A says that evaluation rollouts are added to the prior dataset. Does the final evaluation use new configurations, or ones already seen in this data?
3 comments

We thank Reviewer 5aRh for their constructive feedback. We kindly refer the reviewer to our general response for the comparison with Ambient Diffusion Policy and its conceptual difference from NGD (items 1 and 2), and to the extended real-robot experiments that evaluate a fixed threshold on held-out settings (item 3). We also clarify the listed concerns below:

  • On the Ambient Diffusion Policy (AmbientDP) baseline (W1, Q1): For a fair comparison, we implement AmbientDP [1] using the same policy backbone, training data, and evaluation protocols (as of today, their code is not yet released). Similar to NGD, we sweep the noise threshold over a range of values for AmbientDP and select the value that yields the best average performance on each simulated benchmark. The results below show that NGD effectively outperforms AmbientDP on 6 out of 7 simulated tasks. We have updated Table 1 to include this comparison and provided further details in Appendix B. We will compare our implementation with the authors’ after their code is released, and will report results on all benchmarks in the final version.

    CanSquareTransportTool HangCoffeeBox CleanupCanSorting
    AmbientDP73.8 ± 1.748.4 ± 0.869.8 ± 5.429.2 ± 1.550.7 ± 7.432.3 ± 1.073.8 ± 1.0
    NGD77.7 ± 1.456.7 ± 2.775.1 ± 2.542.4 ± 3.550.8 ± 3.443.5 ± 6.884.3 ± 1.3
  • On the threshold selection (W2, Q2): Table 1 provides the per-task breakdown of the evaluations summarized in Figure 2, reporting both the ungated baseline (τ=0\tau=0) and NGD with the best benchmark-level threshold (τb\tau_b) evaluated following the same protocol. These results therefore reuse the same evaluation runs as in Figure 2. To further demonstrate the robustness of our approach with a fixed noise threshold under held-out settings, we extend our real-robot experiments with additional evaluation configurations and a new deformable manipulation task using 200 externally collected prior trajectories from DROID [2], a large in-the-wild manipulation dataset. With a fixed noise threshold of 0.6 used in all of these experiments, our approach obtains consistent and effective improvements over baselines, as shown by the results below. We also refer the reviewer to item 3 of the general response above for experimental details and full results.

    Success RateJoint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
    CoffeePod & Gear (Extended)65.7%60.0%82.9%
    Drawer Cleanup (Extended)45.7%62.9%80.0%
    Towel Hanging48.0%48.0%76.0%
  • On sampling within the gated region (Q3): For each prior sample, we first choose the action- or state-prediction region with equal probability and uniformly sample noise levels within it. Each loss is averaged over all expert and prior samples it supervises within a batch. Each sample contributes equally to the corresponding loss, regardless of how τ\tau varies. We have added this detail in Appendix B.

  • On prior data collection and environment initialization (Q4): As the evaluation happens periodically during training, the 48 initial environment configurations in Robomimic are randomly initialized in every evaluation using random number generators, which depend on the training seed and the evaluation interval (i.e., the number of training steps between two consecutive evaluations) in our setting. The same prior dataset is shared between all baselines and NGD. We also keep training seeds and evaluation intervals consistent across all methods for fair comparison. Prior data collection uses different random number generators than those used in our evaluation. Therefore, the final evaluation uses configurations not seen in the prior data. We have included this detail in Appendix B.

[1] Wei et al. Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics. CoRL, 2026

[2] DROID Dataset Team. DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset. RSS, 2024

I thank the authors for the response and the new experiments. They address my questions 3 and 4.

For W1/Q1, I appreciate that you added AmbientDP, but the results raise a new question. On Robomimic, your best AmbientDP (55.3 average) matches expert-only DP (55.3), and across the seven simulated tasks it is below DP (SC) on 6. Why does it gain nothing from the prior data, given that the Robomimic prior rollouts include Gaussian action noise, which is a setting AmbientDP is designed for? What are the results with the low-noise interval enabled?

For W2/Q2, you confirm that Table 1 reuses the sweep's evaluation runs. I second Reviewer xcA2 and ask for results with a fixed τ = 0.6 on all simulated tasks, next to the ungated model.

AuthorsOpenReview ↗

Response to Reviewer 5aRh

We thank the reviewer for the follow-up questions and are glad that our initial response addressed Q3 and Q4. We provide additional results and clarifications for the remaining concerns below:

AmbientDP performance on noise-perturbed data

In addition to the local Gaussian action noise, our Robomimic prior rollouts are globally suboptimal, as they come from imperfect policies and many of them end in task failure. This differs from the "noisy trajectories" setting in the AmbientDP paper, where the suboptimal data are jittery, non-smooth trajectories from a sampling-based planner (RRT) that remain globally correct and are suboptimal only locally. On Robomimic, AmbientDP outperforms DP (Naive Pooling) but matches expert-only DP. This suggests that, in our setting, restricting prior samples to the high-noise range suppresses their harmful behavior but does not necessarily extract additional useful information from them. We conjecture that Source Conditioning provides more benefit by explicitly separating the prior behavior mode from the expert one, while still letting useful motion from prior data inform expert-conditioned generation through the shared backbone.

Low-noise interval of AmbientDP

We performed preliminary experiments to study whether AmbientDP can leverage local useful behavior in the low-noise range. To focus on low-noise intervals (tmaxt_{max}, 1], we fixed the high-noise interval [0, 0.05) and varied the low-noise threshold tmaxt_{max} from {0.8, 0.9, 1}. tmax=1t_{max}=1 denotes settings with low-noise interval disabled. We evaluated three challenging simulated tasks: Tool Hang, Box Cleanup, and Coffee, and the results are reported below. We observe that in our setting, enabling the low-noise interval does not improve performance. On Tool Hang and Coffee, it degrades performance with lower tmaxt_{max}. We therefore disabled the low-noise interval and only swept through the high-noise threshold for our reported numbers in Table 1. We will also compare our implementation with the official one once it’s released and will provide a more thorough study of AmbientDP on our evaluation benchmarks.

tmaxt_{max}0.80.91 (low-noise disabled)
Tool Hang16.925.834.6
Box Cleanup28.521.028.0
Coffee45.547.548.5

Robomimic results with τ=0.6\tau=0.6

For DexMimicGen and real-robot experiments, we used τ=0.6\tau=0.6 in our initial manuscript. We have updated Table 1 with Robomimic results achieved at τ=0.6\tau=0.6 and report the full results below for the reviewer’s convenience. Although the average performance at τ=0.6\tau=0.6 is slightly lower than at τ=0.7\tau=0.7, τ=0.6\tau=0.6 achieves competitive per-task performance and the highest average performance in the updated Table 1.

Success RateCanSquareTransportTool Hang
NGD (τ=0\tau=0)75.9 ± 1.156.5 ± 1.270.6 ± 2.339.7 ± 3.6
NGD (τ=0.6\tau=0.6)77.3 ± 1.355.6 ± 1.675.7 ± 0.242.0 ± 1.3
NGD (τ=0.7\tau=0.7)77.4 ± 1.456.7 ± 2.775.1 ± 2.542.4 ± 3.5

We hope these additions help address the reviewer's remaining concerns, and we would be happy to provide further clarification if any questions remain. Thank you for your time and thoughtful feedback!

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive feedback. We have updated the revision and marked the changes in blue. Below, we summarize our findings from the newly added experiments, as requested by the reviewers, and respond to their shared concerns:

  1. On the conceptual differences between NGD and Ambient-style prior approaches: Unlike X-Diffusion[1] and Ambient Diffusion Policy[2], our approach neither restricts the learning of suboptimal samples at high-noise diffusion timesteps nor relies on distributional indistinguishability for supervision from prior experience. Instead, noise thresholds in NGD determine how much information the conditioning variable must retain before prior data can supervise prediction of the other variable in a joint state-action model, which are used to select learning objectives across the two-dimensional noise plane. We have updated the related work to make the conceptual difference explicit.

  2. Ambient Diffusion Policy Baseline: We implement Ambient Diffusion Policy (AmbientDP) using the same diffusion policy backbone, training data, and evaluation protocols in our experimental settings. Similar to NGD, we sweep over a range of noise thresholds for each benchmark and report AmbientDP results obtained using the best noise threshold below. NGD effectively outperforms AmbientDP on 6 out of 7 tasks. We have included the results in the revised Table 1 and provide further details in Appendix B. We will compare our implementation with the official one after it’s released and will report results on all benchmarks in the final version.

    CanSquareTransportTool HangCoffeeBox CleanupCanSorting
    AmbientDP73.8 ± 1.748.4 ± 0.869.8 ± 5.429.2 ± 1.550.7 ± 7.432.3 ± 1.073.8 ± 1.0
    NGD77.7 ± 1.456.7 ± 2.775.1 ± 2.542.4 ± 3.550.8 ± 3.443.5 ± 6.884.3 ± 1.3

[1] Pace et al. X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations. ICRA, 2026

[2] Wei et al. Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics. CoRL, 2026

AuthorsOpenReview ↗

General Response (Cont.)

  1. Extended Real Robot Experiments: We provide additional real-robot experiments to further demonstrate the robustness of our approach with a fixed noise threshold. Rather than using prior trajectories collected under our own robot setup, we instead use 200 demonstrations from DROID [3], a large in-the-wild robot dataset collected across diverse tasks, environments, and camera conditions. Throughout all the experiments, we apply the fixed noise threshold of 0.6, a threshold that provides competitive performance across benchmarks in our initial experiments. Since DROID was not used in our initial noise threshold search, this experiment therefore evaluates not only whether NGD can utilize externally collected prior experience, but also whether a fixed threshold transfers to a new prior-data distribution. We evaluate CoffeePod & Gear and Drawer Cleanup on the original evaluation set with 20 initial object configurations, and extend the evaluations in two ways: (1) we include another 15 initial object configurations as an additional evaluation set for CoffeePod & Gear and Drawer Cleanup ; (2) we introduce Towel Hanging, a deformable-object manipulation task in which the robot must grasp a towel from the table and mount it on a rack, and evaluate the task with 25 trials.

    CoffeePod & GearJoint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
    Original Evaluation Set65.0%65.0%85.0%
    Additional Evaluation Set66.7%53.3%80.0%
    Overall65.7%60.0%82.9%
    Drawer CleanupJoint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
    Original Evaluation Set50.0%65.0%80.0%
    Additional Evaluation Set40.0%60.0%80.0%
    Overall45.7%62.9%80.0%
    Towel HangingJoint State-Action Model (Naive Pooling)NGD (τ=0\tau=0)NGD (τ=0.6\tau=0.6)
    Success Rate48.0%48.0%76.0%

    NGD consistently improves performance across all extended evaluations, with nontrivial gains from noise gating over the ungated variant, NGD (τ=0\tau=0). These results demonstrate that a fixed threshold remains effective across a new source of prior experience, additional evaluation configurations, and a new target task, highlighting the robustness of our approach. We have incorporated these extended evaluations in Section 4.5 of our revision. We also provide real-robot evaluation videos on an anonymous website: https://ngd-rebuttal.pages.dev/ and additional qualitative comparisons in the revised Appendix G.1.

[3] DROID Dataset Team. DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset. RSS, 2024

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.